The Three Accuracy Terms of Magnetostrictive Sensors: Resolution, Repeatability and Non-Linearity

In displacement-sensor selection, resolution, repeatability and non-linearity are often treated as the same thing, and "resolution" is most easily equated with "accuracy". In fact the three describe different error sources: resolution characterises the smallest position increment the sensor can distinguish; repeatability characterises the scatter of readings when returning to the same physical position; non-linearity characterises the maximum deviation between the full-stroke reading and the ideal straight line. This article explains each parameter in turn against measured figures for Germanjet magnetostrictive displacement sensors, and notes what to watch when specifying.

Resolution: the smallest distinguishable position increment

Resolution describes the smallest change of position the sensor can recognise. For example, at 1 μm resolution a position change of less than 1 micrometre may not appear in the reading. Germanjet magnetostrictive displacement sensors measure distance from the propagation time of a torsional stress wave in the waveguide (position = wave speed × time difference), so resolution is essentially set by timestamp accuracy. Datasheets describe this as "Infinite resolution", meaning that theoretical resolution, limited by the timing circuit, can be extremely high; production units are offered in selected steps of 1 / 2 / 5 / 10 / 20 / 50 / 100 μm.

It must be clear that high resolution is not the same as high measuring accuracy. Resolution only answers "how fine can it see"; it neither guarantees that each reading falls at the same position, nor how large the deviation from true value is. A sensor with 1 μm resolution but 0.1%FS non-linearity may be reading a "precise error". Resolution steps should therefore be chosen from the smallest distinguishable increment the control loop needs, and should not be the primary criterion of accuracy.

Repeatability: scatter when returning to the same position

Repeatability is defined as the maximum deviation among readings from repeated measurements at the same physical position. It measures consistency, not "absolute accuracy". Repeatability of Germanjet magnetostrictive displacement sensors can reach the order of ±0.002 mm (2 μm), and is better still on short-stroke models.

In closed-loop control, repeatability is often more important than resolution. The core task of a servo or hydraulic loop is to "return to the target position": whenever the controller issues the same setpoint, it expects a stable, consistent position reading so that it can compute position error and adjust the actuator. If repeatability is inadequate, readings scatter when returning to the same point and the loop will show low-frequency oscillation or positioning drift. Resolution only affects the visibility of tiny changes; repeatability decides whether the system is "stable".

Non-linearity: maximum deviation from the ideal straight line

Non-linearity describes the maximum extent to which the sensor's full-stroke reading, relative to true position, departs from the ideal fitted straight line, usually expressed as %FS (percentage of full scale). Typical non-linearity of Germanjet magnetostrictive displacement sensors is < 0.02%FS (using a 90 mm measuring-range segment as an example). Because %FS is scaled to full scale, the corresponding absolute error is smaller on a shorter measuring range for the same figure.

Because the figure is a percentage of full scale, it must be read together with measuring range: the same 0.02% non-linearity corresponds to ±0.02 mm on a 100 mm range and to ±0.8 mm on a 4000 mm range. Therefore non-linearity is meaningful only when given together with measuring range. The absolute error of a long-stroke sensor is naturally larger than that of a short-stroke unit; that is a difference of range base, not a drop in product performance.

Comparison of the three figures

What to watch among the three parameters when specifying

• Do not use resolution as the sole criterion: the resolution number is usually the most eye-catching, but it only characterises distinguishing ability. What really affects closed-loop positioning stability is repeatability and non-linearity; take these two as the main criteria for positioning applications.
• Non-linearity must be read together with measuring range: "0.02%FS" and "±0.02 mm" do not mean the same thing. Confirm whether the figure is relative to full scale or an absolute millimetre value, and convert to absolute error for the actual stroke.
• Manage absolute-error expectations on long strokes: even at 0.02%FS, a 4000 mm range still has ±0.8 mm absolute error — a mathematical necessity. If tighter is required, consider a short-stroke model or segmented calibration.
• Repeatability depends on installation conditions: the stated ±0.002 mm is mostly based on laboratory conditions. Site vibration, temperature drift and concentricity error between magnet and waveguide all degrade it. Only with installation and alignment in place (see the Series 17 hydraulic-cylinder integrated type mounting notes) can the stated value be reached.

Secondary effects of temperature drift and response time

Besides the three figures above, two further factors affect effective accuracy indirectly. The first is temperature drift: a change in operating temperature alters waveguide speed of sound and material length, causing zero and span to drift; high- and low-temperature duty must allow for compensation or site calibration. The second is response time (update rate): the update rate of Germanjet magnetostrictive displacement sensors varies with measuring range, about 0.5 ms (1200 mm) to 5 ms (7600 mm); in high-speed reciprocating applications, confirm that the update rate meets the control-cycle requirement, otherwise the loop will use position data from the previous sample period.

If the machine uses an analog interface, both the Series 191 analog and the Series 13 mobile hydraulic type can cover it; for multi-axis digital communications and built-in diagnostics, see the Series 194 CANopen or the Series 197 EtherCAT. When selection is uncertain, combine the product selection approach with the operating principle.