Anecdotally, I have to say that this seems right. At least two of disallowed tries scored by the Cats on Sat seemed legit to me. I've been to Gilbert Brutus a couple of times in the past year and always felt that the rub of the green was against the french side. I'd not thought much of it, but perhaps it's true. Obviously without reading the methodology of the study (by a frenchman it should be noted), it's impossible to say one way or another.

You can collect statistical data from anything and find trends and patterns in it. It is the nature of statistical data.

That does not mean the statistical data has any value in determining cause and effect in actual events, unless it is grounded in a corroborating analysis of those actual events.

i.e. - if you match penalty counts against nationality of the referee, it will either show refs give more penalties against their own nation, or it will show they give less. Identical results are most unlikely, though variations may be very small or, for whatever reason (correlation or coincidence) the variation might be quite large.

It's interesting you say "After allowing for variables such as the quality of opposition, which team was at home, et cetera"... this highlights some of the problems in drawing conclusions from such statistical data. What are the effects of those variables? Does a higher-quality opposition concede less penalties because they are better (and so higher in the league), or is a team higher-placed in the league there because they concede less penalties and so win more games? Does being at home increase your penalty count, or does it decrease it? Are referees sympathetic to home crowds, or antagonised by them? And will the decisions and patterns of behaviour of one official make the sllightest difference to the patterns from another? In other words, how can you isolate one factor from another... where's the evidence in the study? and where's the bias in the interpretation of results?

Stats will always subject to interpretation and presentation, but you cannot evaluate the decisions of an official based on stats and averages. You cannot, in fact, do it without observing the performance of the official on individual decisions in individual games.

If I looked at the penalty count against Lancashire teams and against Yorkshire teams, it would either be higher or lower. If it was higher, I could say Lanky sides were unfairly punished by the Leeds-based RFL. If it were lower I could say that referees lean west. Or I could say that the higher proportion of good Lancashire teams (Wigan, Saints, Wire vs Salford, 3-1 high-quality vs poor quality) means that their sides have better discipline than the more evenly-split Yorkshire teams. Or I could say that the higher rainfall in western England means matches are slower and less open, so there is less time in which to concede penalties, and there are fewer offsides because there are fewer quick ptb's in heavier conditions.

But - and here is the most important point about my hypothetical study, and about the real one mentioned in the Independent - it would all be spurious, made-up b0llocks.

...or, to put it another way, you cannot draw conclusions about any bias of referees unless you judge whether their decisions are correct or not.

You can use statistics to draw inferences without looking at individual decisions. If, when refereeing games between Teams A & B, one particular referee P gives more penalties to A than B, and does this every game between them, season after season, whereas referee Q always gives more penalties to team B, then it is reasonable to conclude that at least one of P & Q is biased or that they are interpreting the rules differently.

Incidentally, I'm not arguing that this study is correct in its conclusions. If anyone with a background in statistics wants to look up the paper and comment I would like to know their views. But you can't dismiss the findings simply because the authors know nothing about rugby.

No it isn't. Your data set is nothing like large enough.

I've just tossed a coin 10 times. The results were H - T - H - H - H - T - H - H - T - H.[this is actually true!] From this I conclude that Heads is twice as likely to come up as tails, and that my coin is intrinsically biased to land tails down.

When dealing with micro data-sets, like, say, the performance of one ref officiating the same combination of teams over several years, statistical extrapolation doesn't work. Examining the production of those data-sets is the only useful way forward, especially when considering a subjective performance (decisions) against an objective benchmark (rules of the game).

I didn't state how large the dataset would be. I was only trying to illustrate that you didn't need to look at individual decisions to determine whether there may be bias. Obviously, if the data set only included a few games then random chance could explain the differences. But if over dozens of games between the same two sides, refereed by one of the two referees, one always favoured one team and the other favoured the other team then that would indicate bias (or different rule interpretation). The more games the pattern continued, the more likely to be true this would be.

As I have stated, I don't have the expertise to analyse the statistics used in the published paper. I presume it was peer-reviewed by a statistician, but accept that this is not a cast-iron guarantee that the analysis does justify the conclusion. However, if you want to disagree with the conclusion you have to read the paper, look at the methods used, and base your disagreement on that. Simply stating that the authors know nothing about rugby so don't know what they are talking about is not a valid refutation of the paper.