A lesson relearned
Dr. Robert M. Gresham, Contributing Editor | TLT Lubrication Fundamentals April 2019
When testing oil and grease, there’s a big difference between repeatability and reproducibility.
© Can Stock Photo / TonoBalaguer
I had the pleasure of attending the OilDoc Conference in Rosenheim this past January. As usual, the conference was well attended, provided a good opportunity to reconnect with a number of our European friends, as well as those from the U.S., and there were a number of very good presentations.
One such presentation was by STLE-member Steffen Bots with Lubeserve in Germany, who has served on STLE’s certification committees and is now a new member of our board of directors. Steffen’s presentation, Global Used Oil Analysis Programs: Challenges and Solutions, reminded me of lessons learned in graduate school, at DuPont and E/M Lubricants having to do with testing and test results.
Steffen’s point was that with used oil analysis you might not get the same answer you get with fresh, unused oil because you could be introducing bias into the test method or some other source of inaccuracy or variability. Too often we tend to think of the results that we get from a test as an absolute value, but it can be anything but. Let’s take a look.
Accuracy has to do with the question, did you get the correct measurement value? Bias is a systemic error that contributes to the difference between the mean of a large number of test results and an accepted reference value. The accepted reference value is considered the true value and, therefore, bias has to do with the degree to which that true value is not obtained. In general, through calibration, bias can be reduced or eliminated. You can see Steffen’s point that with used oil there is no accepted reference value or calibration.
Precision has to do with the variability of the test method itself. Does the method provide the same number every time, or does it vary, and, if so, by how much? In ASTM parlance, precision is stated in their published standards as the repeatability and the reproducibility of the method.
Repeatability has to do with the difference between two test results performed by the same operator in the same laboratory with the same apparatus on the same test material under constant operating conditions.
Reproducibility has to do with the difference between two single and independent results obtained by different operators working in different laboratories on identical test material in the normal and correct operation of the test method. When analyzing used oil, is there an impact on repeatability or reproducibility?
While I tend to get on a soapbox because of a bad experience, the method for measuring consistency of greases, ASTM D217, provides a good example of the inherent variability in testing. It states that the method’s worked penetration variability for greases like a No. 2 for more than one lab and operator to be 20 units. In a given situation test results could vary by 40 units (+/-20) and statistically be the same result.
Thus, when a specification is written and/or sales contract agreed to, or when analyzing used oil, these are important considerations. The following story is an example.
I once had a grease customer who wanted the consistency of the grease we sold him to be very consistent (pun intended). Grease consistency is a somewhat ethereal word to describe the apparent thickness, soupiness or whatever of the grease. ASTM D217-10 describes the test method of measuring consistency in greases and provides a way of categorizing this property (see Figure 1).
Figure 1. Categorizing the consistency in grease with NLGI grade.
I suppose a very inconsistent grease is like a brick, and a very consistent grease is like a thick fluid—since the numbers go up—but that seems counterintuitive. I digress.
Anyway, people refer to grease consistency by its National Lubricating Grease Institute (NLGI) ranking. As I recall, the customer specified the grease at 305 +/-5. The product wasn’t much; we merely mixed an inert solid lubricant into base grease that we purchased according to his specification. The base grease was an NLGI No. 1. As you can see, a No. 1 can vary from 310-340, but what about a 305? After we mixed the additive, the consistency decreased a little (got thicker) and was on the low end of a No. 1.
However, to address the issue of how consistent the consistency of our grease was, one should look to the ASTM method D217-10 concerning the variability of the test method itself (see Figure 2).
Figure 2. ASTM D217-10 states that the method’s worked penetration variability for more than one lab and operator is 23 units.
It states that the method’s worked penetration variability for more than one lab and operator is 23 units. Thus, we—the customer and I—could vary by 46 units (+/-23) and statistically be the same result. As you can guess, with a specification of +/-5, we could test the same batch and sample of the grease, and at least 82% of the results would be out of specification. Thus, the customer could and did frequently reject a lot and send it back. We could have simply relabeled it (which I did a few times), sent it back and 23% of the time it would have met the specification. If this sounds like a recipe for disaster, it is only because it was.
Both customer and vendor quality assurance departments and/or outside test labs need to be able to generate the same numbers on the same samples within the confines of the test methods being employed. Specifications need to reflect the reality of our ability to test with precision and accuracy. And, as Steffen points out when you change the inherent nature of the material being tested, used oil versus fresh oil, you need to be cognizant that you might be introducing bias into the test method.
Thanks, Steffen, for helping me relearn an old lesson.
Bob Gresham is STLE’s director of professional development. You can reach him at rgresham@stle.org.