“Facts are stubborn things, but statistics are more pliable.” — Oscar G. Foellinger, 1927
As an engineering student, I once used empirical data to solve a design problem. The approach gave a reasonable answer, but my professor scolded me. “Mere curve fitting,” he sneered. “Good engineering requires working from first principles.”
I didn’t agree then, I don’t agree now. So, when a statistician scolded me for using the “n plan” when sampling during a PSM compliance audit, I was put out, but not put off.
The n plan
Bluefield Process Safety has historically used the “n plan” to determine the number of items to review during an audit. The “n plan” is based on looking at a random sample from the population being audited. The number of samples, n, is a function of the number of items in the population being audited, N, where n = 1 + √N, rounded to the nearest integer.
In the 1920’s, Harold F. Dodge, who worked at Bell Labs with Walter Shewhart and others, developed a way to sample product from an unknown source: randomly choose a quantity of samples equivalent to the square root of N, plus one, where N is equal to the total number units in the lot. Many companies and federal agencies—USDA, FDA, DoD—used this method. Some companies still use this method. The “n plan” has the advantage of being a simple plan to remember and implement. How hard is it to remember “the square root of N plus one”? The calculator on a cellphone can do the math without any special functions.
Not Statistically Based
However, the n plan is not statistically based. One review concludes with the advice that “…this rule should not be used to select a sampling plan to infer a population defective rate.”
The World Health Organization advises that “[i]t is important to recognize that the ‘n plan’ is not statistically based and should be used only as a guiding principle” and that “the ‘n plan’ should be used with great caution and only when the material to be sampled is considered uniform and is supplied from a recognized source.”
ANSI/ASQ Z1.4
Instead, the sampling approach described in ANSI/ASQ Z1.4 is now frequently used instead.
Z1.4 is not difficult to use. It relies on a set of statistically based tables. The auditor must determine which of three “general inspection levels” to use: Level I for a subject that is expected to pass the audit, Level III for a subject that is not expected to pass the audit, and as a default, Level II for which the auditor has no expectations.
Then, having determined the sample size based on the general inspection level and the number of objects in the subject to be audited—operating procedures, equipment files, inspection reports, training records—the auditor examines each object and determines if there are discrepancies. The acceptable number of discrepancies is in another table in Z1.4.
It is not difficult to do. It just requires that you have access to the tables and know how to use them.
A Comparison of Sampling Plans
Torbeck compared the sample sizes suggested by the n plan and by Z1-4. “Sample sizes for Sqrt(N) + 1 are very close to the sample sizes in Z1.4 General Level I. For small lot sizes, Sqrt (N) + 1 samples are actually larger than Z1.4 General Level I.” For a statistician used to looking at populations of thousands or even millions, the typical population for a PSM compliance audit is miniscule. So, let me add that for a typical population in a PSM compliance audit, even Z1.4 General Level II is exceeded by the n plan.
Torbeck again: “Both Z1.4 General Level I and Sqrt(N) + 1 are universally used in the industry for attribute sampling and inspection. … We can have as much confidence in Sqrt(N) + 1 as we do of Z1.4 … Any comments or publications to the contrary are incorrect.”
What About Findings?
Z1.4 uses a table to inform the auditor of how many discrepancies constitute a finding. For minor discrepancies, the table conforms to a rule that if 10% or more of the samples have minor discrepancies, it is a finding. What is a minor discrepancy? The record, report, file, etc. exists, but is incomplete. For major discrepancies, where the item simply does not exist, the Z1.4 table conforms to a rule that if 6% or more of the samples are missing, it is a discrepancy. Given the likely population size during an audit, this will often mean that one discrepancy in the sample set results in a finding.
Mere curve fitting, but it works.
Let’s Be Practical
From a practical standpoint, the-n plan works for the population sizes examined during audits. It is easy to remember and easy to explain. Such simplicity, whether statistically based or not, is valuable to an auditor and to the personnel charged with addressing the findings. If the n plan is what you use, keep using it. We will. If it is not, knock yourself out.
In the end, though, the purpose of an audit is to identify areas that need improvement, and either method will do that. Use what works. Leave the bickering to another place, another time.
