Performance of CMMs: Testing, Calibration, and Uncertainty

1. Introduction
Coordinate measuring machines (CMMs) are the most advanced dimensional measuring instruments on the market today. Their unlimited versatility allows them to be used for an infinite number of measurements. But the inherent flexibility of CMMs can also make them difficult to implement. If you are new to CMMs, you are probably wondering how to determine which of the many available machines will be accurate enough for your needs and also how to best compare two similar looking machines. Once a CMM is installed, the question of how to calibrate a machine with so many capabilities is often asked. And now, with many new quality initiatives being pushed onto manufacturing and calibration, the big question is how to estimate the uncertainty of measurements being made using CMMs.

We will try to answer all these questions here. We will first discuss performance testing of CMMs and how to best use these tests. Then we will discuss what it means to calibrate a CMM and what to look for when someone arrives to calibrate your machine. We will also discuss interim tests, which are simple tests that you can run between calibrations to ensure your CMM is still in tolerance. Finally, we will discuss measurement uncertainty and give direction on how you can use all the other stuff, i.e. the performance tests, calibration, and interim testing, to get a good handle on the uncertainty.

2. Performance Testing
CMMs have been around since the late 1950’s, but when standardized performance tests hit the market in the 1980’s, the quality of machines increased significantly. The primary purpose of performance tests is for improved commerce. If you are looking to buy a CMM, you can easily scan the specifications from the different CMM manufacturers and compare tolerances for the various performance tests. Since the standardized testing procedures are well documented, you can ensure that you are truly comparing “apples to apples”. For this reason, some CMM manufacturers either improved their quality or got out of the CMM business when the standards were released.

The performance test specifications are also part of the machine quote and become your guarantee that you are indeed getting the machine you paid for. However, since increased machine accuracy usually comes at a higher cost, it is important for you to understand what the tests mean. Do you check the size of the engine in your new car before driving it home? Sure you do, and it only takes a minute to ensure that the optional V-8 you just paid extra for is really under the hood. If you don’t check or don’t know what to look for, then there is no telling what you might actually be driving. The same is true for CMM performance tests – learn the tests before you buy a machine and make sure your machine meets the specifications before you send in that final payment.

2.1 History of Performance Tests
We are primarily going to discuss two different performance tests: (1) the international standard, “ISO 10360”, and (2) the U.S. standard, “ASME B89.4.1”. We will also briefly review the German “VDI/VDE 2617” test, as it has been used a lot in the past. In addition, we will highlight some of the other test procedures that have been used over the years, but the ISO, ASME, and VDI/VDE tests are the important ones. All of these major CMM standards are still in use internationally for CMM commerce.

The reason there is more than one internationally accepted standard is mostly political, but we won’t bore you with that here. In the end, there is more than one that you may come across. In the U.S. market, both the ISO and ASME standards see a lot of use, and some large companies prefer the use of one to the other. Most international CMM manufacturers will commonly provide specifications to all the major standards in their literature, but more and more are starting to just use the ISO standard. So, let’s start by discussing this ISO standard.

2.2 ISO 10360
The ISO 10360 standard is really a series of standards. The most important part of the series is Part 2, which was first published in 1994. The official designation is ISO 10360-2:1994. As with all standards, the part number (2) and the date (1994) are both very important as the standard could, and will, change over time. The ISO standard is the youngest of all the CMM standards, but since it is the ISO standard, it is fast becoming the most popular, both in the U.S. and worldwide.

10360-2 has two separate tests. The first is the length measuring performance, designated as “E”, and the second is the probing performance, designated as “R”. The E test is a complete test of the CMM to measure length, an important fundamental characteristic of a machine. The test procedure calls for a series of measurements of either calibrated gage blocks or a step gage. If you haven’t seen one before, a step gage is a unique gage with a series of linear steps. See Figures 1, 2, and 3. Step gages have long been popular for checking machine tools and measuring instruments, and they are particularly useful for testing CMMs.

Figure 1. Step gage close-up.

In the ISO E test, at least 105 length measurements are performed across seven different positions. The seven measurement positions can be chosen at the discretion of the customer, per the standard, but manufacturers will usually recommend three positions be parallel to each of the three machine axes (X,Y,Z) and the other four be the body diagonals. The body diagonals usually show the worst machine errors, and the runs parallel to the machine axes show how good each axis is independently. A step gage being used for these measurements is shown in Figures 2 and 3.

Figure 2. Checking Y-axis with step gage.

Figure 3. Body diagonal measurement.

The method of specifying the manufacturer’s tolerance for E can often be confusing for those new to CMMs. The most common method is the use of an equation that looks something like

MPEE = A + L/K μm

where MPE is the maximum permissible error, a relatively new term being used in ISO standards today (note that just E, not MPE, is also used). In other words, MPE is the tolerance. The A and K terms are constants supplied by the manufacturer, and L is the measured length in millimeters (mm). For a typical machine, the specification might look like

MPEE = 2.5+3L/1000 μm.

If this equation is applied to a few different lengths, you might see results like those listed in Table 1. As shown, value calculated using the equation.

To verify a CMM meets its specification, a minimum of five length measurements are made three times at each of the seven positions (that’s 105 total). The measured lengths are then compared to the calibrated values of the step gage and deviations must be less than the machine specification for all 105 measurements. An example of one of the seven runs is shown in Figure 4.

The E test is a thorough system test of the CMM and is sensitive to machine geometry and scale errors, repeatability, and some probing errors. As will be discussed later, E is a key test in establishing traceability for your CMM. Since E is also sensitive to thermal variation, it is always specified for a particular temperature range. Machines equipped with automatic temperature compensation usually allow that range to be extended.

E is not sensitive to all probing errors, and therefore the probing test R is also used. R is a simple and quick test that uses a calibrated precision sphere. Twenty-five points are taken across half the sphere, and the measured form is reported.

The setup for performing the R test is shown graphically in Figure 5. This test is sensitive to any directional measuring problems with the probing system. Though this test does not isolate probe problems, it is very good at finding any random or systematic errors with the probe, including the well known “lobing error” that occurs with some touch trigger probes.

E and R are currently the only tests for which you will generally find specifications when using ISO standards. Recently, additional parts have been published. These parts include:

ISO 10360-1:2000, Vocabulary ISO 10360-3:2000, Rotary Table ISO 10360-4:2000, Scanning ISO 10360-5:2000, Multiple-stylus

These parts are still too new to expect all manufacturers to have specifications available in their literature. They also apply to only particular machines and not to all situations. However, if they apply to your machine, you should study these tests as well.

Of all the new ISO tests, the one that is receiving the most interest in the CMM community is Part 4, the scanning test. As high-speed contact scanning is becoming more and more popular on CMMs, this new test should be very useful. The scanning test is similar to the R test in Part 2, but involves scanning the sphere instead of taking only single points. A graphic of this test is shown in Figure 6, where the paths A, B, C, and D define the scan measurement path. In addition, the scanning test is a timed test. Since scanning performance is directly related to the measurement speed, it is important that the time for this test is also included in machine specifications.

Figure 6. The new ISO 10360-4 scanning test.

There are also plans for a new release of Part 2 sometime in late 2001. The only expected difference between the current 10360-2:1994 version and the new version will be the handling of measurement uncertainty. In current ISO thinking, uncertainty must be accounted for when proving conformance. This may sound like a minor difference, but it could have a major impact on the E test. For now, a word of warning – be certain you know the date of the standard being referenced in specifications and quotes, as the values may be different between the two versions. Under the new version, E specifications may increase.

2.3 ASME B89.4.1 (revised B89.1.12)
The U.S. standard for testing CMMs is analogous to the current use of the inch system over metric in the U.S. It works just fine, but it is very different than the standards used outside the U.S. Originally published as ASME/ANSI B89.1.12 in 1985, this standard changed its name with the 1997 publication, but the heart of the test procedures have not changed over time.

The biggest difference between the ASME and ISO standard is the total number of tests. Where ISO has one machine system test, E, there are five tests in the ASME standard, each one being sensitive to different machine errors. These tests include:

  1. Repeatability
  2. Linear Displacement Accuracy 3. Volumetric Performance
  3. Offset Probe Performance
  4. Bidirectional Length

The advantage of having multiple tests is that you get potentially more useful information, as certain measurements you make will be influenced by some of the errors but not others.

The disadvantage of the ASME standard is that you need to know all the tests, and the manufacturer needs to specify them all. This is actually a big problem with the ASME standard. Most commerce using this test standard is unfortunately based solely on the Volumetric Performance test, often called the “ball bar” test. See Figure 7.

The Volumetric Performance test involves measuring the length of the ball bar as the distance between its two end spheres. This measurement is repeated in many positions throughout the machine volume. Any change in the measured length reflects machine geometry errors. Though this is a good test, repeatability errors are averaged out and since the ball bar is not calibrated, this test does not provide any traceability. The ASME standard does provide tests to handle these other issues (the Repeatability and Linear Displacement Accuracy tests), but these tests must be used. In addition, any influence of probe tip calibration on size measurement must be tested for separately using the ASME Bidirectional Length test, as none of the other tests are sensitive to this error.

Figure 7. A better ball bar. The dual balls on each end provide a superior mounting method.

If just a single number is listed in CMM literature under ASME, then the manufacturer is probably providing a specification for the ball bar test. Be sure to ask for specifications for all the other tests as well. Since you will need to know more about these tests, here is a short summary of the test procedures:

  1. Repeatability: The center coordinates of a sphere are measured ten times quickly. Only four probing points per sphere are allowed. The range of the coordinates is reported.
  2. Linear Displacement Accuracy: This test is similar to the ISO E test, but measurements are only made parallel to the machine axes. This is the key test for traceability, and either a step gage or laser interferometer is used.