One of the questions that we have received on the support team quite a few times and something that confused me when I started at Agilent is the concept of our resolution supplemental characteristic versus our accuracy specification. I sat down with my colleague Russell and we wanted to do a simple explanation of the differences.
If you look at our power supply offering, there is always an accuracy specification and a resolution supplemental characteristic for both programming and measurement. For the purposes of this blog post, we are going to look at the programming accuracy (0.06% + 19 mV) and programming resolution (3.5 mV) of the N6752A High Performance DC Power Module. Please note that these same explanations apply to the measurement side as well but for the sake of brevity we will be sticking to programming in our example.
Let’s start by talking about resolution. Our power supplies use Digital to Analog Converters (DACs) to take the user inputted settings and convert them to analog signals that set a programming voltage that will interact with the control loop of the power supply to set the output. The resolution supplemental characteristic represents one single count of the DAC. This is also known as the Least Significant Bit (LSB). What this means for our end user is that the smallest step they can make between two settings on the unit is the programming resolution number. In our example, the N6752A can be set to 0.9975 V, 1.001 V, 1.0045 V, etc. These are all multiples of 3.5 mV and any setting that falls between two DAC counts will be put into the nearest count. If the user tried to set the N6752A to 1 V, the power supply will actually be set to 1.001 V since that is the nearest count. This is also known as quantization error.
The accuracy specification always includes an error term for the quantization error. This is typically half of the resolution supplemental characteristic. The accuracy specification also includes many other factors such as DAC accuracy, DAC linearity, offset error of operational amplifiers, gain errors of the feedback loops, and temperature drift of components. The accuracy will always be worse than the resolution since it includes all of the factors listed above as well as the term for the quantization error. You can definitely see this in the N6752A where the resolution is 3.5 mV and just the offset of the accuracy specification not including the gain term is 19 mV which is more than 5 times greater than the resolution.