However, in situations involving numerous low-value items, such as minor but widespread defects, the cost and time required to prove each individual item can be prohibitive or, at least, disproportionate. This is where sampling evidence has been posited as a solution.

Sampling evidence in construction cases has been a means of establishing liability and quantum for some time (see for example Associated British Ports v Hydro Soil Services[1]), but the 2016 case of Amey LG provided valuable and much needed detail on the judicial treatment of proof by sampling in situations where true randomised sampling is not possible.

What is evidence by sample?

There are two broad sampling methodologies relevant to this article: probability sampling, and non-probability sampling.

Probability sampling is defined as a method that identifies a sample population through a randomized process, in which each constituent has a non-zero probability of selection. It is, in principle, independent, objective, and representative, with calculated degrees of confidence to account for any random errors as well as any potential bias. In this sense it can be easily extrapolated for evidential purposes; the sample can be by statistics shown to be representative of the whole.

Non-probability sampling, on the other hand, relies on subjective judgments about which elements are sampled, rather than relying on a random process. Accordingly, the selected population is based on the selecting party’s own subjective judgements as to the appropriateness of their selection. In other words, there is a degree of risk that bias underpins the population selection, which cannot be supported statistically, limiting its extrapolative potential.

In principle, however, non-probability sampling can be used to deduce and apportion quantum and liability. The question when employing non-probability sampling is: how to make it sufficiently representative, in order that a tribunal can rely on the sample?

Considering these definitions, it is easy to appreciate why probability sampling is often promoted over non-probability sampling. A sampling methodology based on randomization, which can be backed up statistically, is easier to justify and brings greater comfort to tribunals in reaching their decisions. However, the factual circumstances of a typical construction dispute rarely allow for a randomized sampling process.

A typical example is a Contractor’s poor workmanship causing widespread defective works. Usually, the Claimant would have to investigate each individual defect to ascertain liability, causation and quantum. This might be practical if investigating comparatively small numbers, but, if those defects number in the hundreds or thousands, the costs can outweigh the potential recovery. Further, traditional probability sampling methods are of little assistance in such circumstances, because as soon as you are looking at specific targeted instances, then the sample can no longer be considered random.

Amey LG v Cumbria County Council

This is where the decision in Amey LG[2] has provided much needed guidance. Amey was contracted by Cumbria County Council to provide highway maintenance. In Amey’s final monthly payment application Cumbria deducted sums for defective works, which prompted legal action from both sides, including a counterclaim by Cumbria for the sum of the defective works.

Cumbria advanced their case based on a sampling method that they claimed to be probability sampling with a confidence rate of 95%. The court accepted that it was appropriate to advance a sampling methodology, as it was impractical that Cumbria should be forced to examine each defect. However, the court disagreed that the sampling was probability sampling (i.e. that it was random), and that it could be supported to a confidence rate of 95%. While the court ultimately dismissed Cumbria’s claim, they did so with the caveat that a non-probability sampling approach was a viable basis on which a Claimant could bring their claim, if following prescribed guidelines.

Here the court emphasized several points that should be considered when undertaking a non-probability sampling analysis:

1. The process should be the subject of advanced planning so that it can be defended.
2. That those conducting and supervising the exercise should take care to ensure that those issues which might arise during the course of the exercise (that might cast doubt on the representative nature of the findings) are identified and addressed.
3. To get as close to probability sampling as reasonably possible, so as to remove “subjective contamination” and thus removing/reducing any potential bias.
4. Finally, as far as it is not possible to remove bias, to be able to identify it and take appropriate steps to address it.[3]

Furthermore, this author suggests that this judicial guidance should be read with the general principles agreed between the parties’ experts in the case:

1. That the relevant population and the relevant unit of analysis should be defined.
2. That the sampling frame that may or may not be identical to the sampling population should be defined.
3. That there should be a procedure to draw a sample from the relevant population.
4. The relevant characteristics for each element in the sample should be measured using a reliable measurement protocol.

Non-Probability Sampling Application in Practice

Following from the above, there is nothing in principle to suggest that a claim based on non-probability sampling cannot be successfully pleaded before a court or tribunal. However, the principles in Amey LG, and thus the above procedure, have yet to be tested as the case has received no judicial treatment to date.

In fact, the only construction case dealing with non-probability sampling that has come before the English courts since Amey LG has been ICI v Merit.[4] This case involved defective welds and overpayments, but made no reference to the procedure in Amey LG. This begs the question: to what extent can we rely on the principles In Amey LG?

Conclusion

Amey LG offered much needed guidance to those in construction disputes facing the prohibitive cost of investigating high-volume, low-value defects on a traditional basis. However, Amey LG’s lack of judicial treatment makes it difficult to determine how cases based on non-probability sampling will be considered in future.

It is unfortunate that Amey LG was able to lay a foundation for the use of non-probability sampling, without giving a good example of its successful use. However, it is still significant that a framework has been laid down, which is valuable to parties pursuing proportionate and effective construction claims. Indeed, similar common law jurisdictions have struggled with how to address these types of claims, though, there is a sense that sampling evidence more generally is slowly becoming more acceptable.

The lack of judicial treatment after four years is concerning to say the least, but it is not definitive one way or the other. ICI v Merit, ultimately,can be distinguished, based upon the sampling analysis and expert evidence, which was clearly biased and easily dismissed. Ultimately, its lack of treatment is indicative of the fact that the law is still struggling to accept claims based on non-probability sampling analysis. This fact alone sends a clear signal to would-be claimants in such circumstances, plead your case as traditionally as possible, and to the extent that this is not possible, the framework in Amey LG should be carefully considered.

[1] Associated British Ports v Hydro Soil Services [2006] EWHC 1187 (TCC).

[2] Amey LG Ltd v Cumbria County Council [2016] EWHC 2946 (TCC)

[3] In this case the judge determined that Amey LG’s statistical sample was biased, both in respect of geography and time (they focused on the last 3 years and on the most heavily trafficked roads), thus their sample was considered un-representative.

[4] Imperial Chemicals Industries Ltd v Merit Merrell Technology Ltd [2017] EWHC 1763 (TCC).