...or, to put it another way, you cannot draw conclusions about any bias of referees unless you judge whether their decisions are correct or not.

You can use statistics to draw inferences without looking at individual decisions. If, when refereeing games between Teams A & B, one particular referee P gives more penalties to A than B, and does this every game between them, season after season, whereas referee Q always gives more penalties to team B, then it is reasonable to conclude that at least one of P & Q is biased or that they are interpreting the rules differently.

Incidentally, I'm not arguing that this study is correct in its conclusions. If anyone with a background in statistics wants to look up the paper and comment I would like to know their views. But you can't dismiss the findings simply because the authors know nothing about rugby.

No it isn't. Your data set is nothing like large enough.

I've just tossed a coin 10 times. The results were H - T - H - H - H - T - H - H - T - H.[this is actually true!] From this I conclude that Heads is twice as likely to come up as tails, and that my coin is intrinsically biased to land tails down.

When dealing with micro data-sets, like, say, the performance of one ref officiating the same combination of teams over several years, statistical extrapolation doesn't work. Examining the production of those data-sets is the only useful way forward, especially when considering a subjective performance (decisions) against an objective benchmark (rules of the game).

I didn't state how large the dataset would be. I was only trying to illustrate that you didn't need to look at individual decisions to determine whether there may be bias. Obviously, if the data set only included a few games then random chance could explain the differences. But if over dozens of games between the same two sides, refereed by one of the two referees, one always favoured one team and the other favoured the other team then that would indicate bias (or different rule interpretation). The more games the pattern continued, the more likely to be true this would be.

As I have stated, I don't have the expertise to analyse the statistics used in the published paper. I presume it was peer-reviewed by a statistician, but accept that this is not a cast-iron guarantee that the analysis does justify the conclusion. However, if you want to disagree with the conclusion you have to read the paper, look at the methods used, and base your disagreement on that. Simply stating that the authors know nothing about rugby so don't know what they are talking about is not a valid refutation of the paper.