Worked example
Example: Default five points, 0.5 % tolerance. The inputs, in the units selected above:
| Input | Value |
|---|---|
| Reference value at point 1 | 0 - |
| Reading rising at point 1 | 0.10000 - |
| Reading falling at point 1 | 0.30000 - |
| Reference value at point 2 | 25.000 - |
| Reading rising at point 2 | 25.200 - |
| Reading falling at point 2 | 25.400 - |
| Reference value at point 3 | 50.000 - |
| Reading rising at point 3 | 50.100 - |
| Reading falling at point 3 | 50.400 - |
| Reference value at point 4 | 75.000 - |
| Reading rising at point 4 | 74.900 - |
| Reading falling at point 4 | 75.200 - |
| Reference value at point 5 | 100.00 - |
| Reading rising at point 5 | 99.800 - |
| Reading falling at point 5 | 99.900 - |
| Calibrated span | 100.00 - |
| Tolerance as a share of span | 0.50000 % |
Applying the formula below gives:
| Result | Value |
|---|---|
| Largest error as a share of span | 0.40000 % |
| Largest rising–falling difference (hysteresis plus deadband) as a share of span | 0.30000 % |
| Result (1 = within tolerance, 0 = fails) | 1.0000 |
Open this example in the calculator and change any input.
Formula
\(e = \max\frac{|x_{read} - x_{ref}|}{S}, \quad H = \max\frac{|d_i - u_i|}{S}\)
Variables
| Symbol | Meaning | SI unit |
|---|---|---|
| \(x_{1}\) | Reference value at point 1 | - |
| \(u_{1}\) | Reading rising at point 1 | - |
| \(d_{1}\) | Reading falling at point 1 | - |
| \(x_{2}\) | Reference value at point 2 | - |
| \(u_{2}\) | Reading rising at point 2 | - |
| \(d_{2}\) | Reading falling at point 2 | - |
| \(x_{3}\) | Reference value at point 3 | - |
| \(u_{3}\) | Reading rising at point 3 | - |
| \(d_{3}\) | Reading falling at point 3 | - |
| \(x_{4}\) | Reference value at point 4 | - |
| \(u_{4}\) | Reading rising at point 4 | - |
| \(d_{4}\) | Reading falling at point 4 | - |
| \(x_{5}\) | Reference value at point 5 | - |
| \(u_{5}\) | Reading rising at point 5 | - |
| \(d_{5}\) | Reading falling at point 5 | - |
| \(S\) | Calibrated span | - |
| \(\epsilon_{max}\) | Tolerance as a share of span | - |
| \(e_{max}\) | Largest error as a share of span | - |
| \(H\) | Largest rising–falling difference (hysteresis plus deadband) as a share of span | - |
| \(\text{pass}\) | Result (1 = within tolerance, 0 = fails) | - |
Method and standards
Definitions.
Where it is used
- Transmitter and gauge calibration records.
- ISO 9001 and audit records.
Common mistakes
- Approaching points from the wrong direction.
- Overshooting a point and coming back.
Assumptions and limits
- Five points; same span for all.
- H is the total difference between falling and rising readings, which ISA-51.1 splits into hysteresis and deadband; this tool does not separate them.
- Pass/fail uses the largest error only; a rising–falling difference above the tolerance is flagged in a note.
References
- Error and hysteresis as defined in ISA-51.1 (process instrumentation terminology)
Questions and answers
What do the default readings show?
Largest error 0.4 % of span at 25 and 50 %, hysteresis 0.3 %: within a 0.5 % tolerance.
What is hysteresis?
The difference between rising and falling readings at the same point.
Why five points?
0, 25, 50, 75 and 100 % show zero, span and linearity errors.
Method and article reviewed by FireCAD engineering on 12 November 2026. Every result is checked against an independent calculation on each release: how we verify our calculators.