Comparing Imager’s Simulated Images to Real Data

Comparing Simulated to Real Data

Comparing Imager’s Simulated Images to Real Data

In this post we compare Imager’s simulated images to actual images acquired with a monochrome IDS uEye machine vision camera and an Edmund Optics 8.0mm micro video lens.  The setup:

  • An Imatest ISO 12233:2014 color test chart, was used as the target and is shown below.
  • The target was illuminated with 3600K halogen lamps.
  • The gray levels of the target used for simulation were adjusted to compensate for the gamma of the printer.
  • The Imager model was configured solely with data taken from spec sheets, and actual measurements, e.g. illumination level and distance to the target.


Imatest ISO 122332014

Fig1: Test chart used in the study


Subjective comparison:

The first image shown below was acquired with an IDS camera and Edmund Optics lens, and the second image is the simulated result from Imager.

Acquired Image Mid Exposure

Fig 2: Image captured with IDS camera and Edmund Optics lens (actual size 752×480)


Simulated Image Mid Exposure

Fig 3: Simulated Image (actual size 752×480)


Noise comparison:

The images above show nearly equal amounts of “graininess” or noise. At the brightness levels that the images were captured, the predominant noise is from photon noise, or shot noise.

The image below shows both the modeled and measured results at a mid-level exposure where the brightest parts of the target are at 200 counts (out of 255), and a low-level exposure where the brightest parts of the target are at 66 counts. In the low-level exposure images, the graininess increases as a result of a shorter exposure time capturing fewer photons. Note that the low-level simulated and acquired images were brightened for easier comparison with the mid-level images.


SNR Comparison

Fig 4: SNR Comparison


The mean and standard deviation of the brightest regions of the 4 images were measured and the signal to noise was calculated to show a quantitative comparison of the images.

ImageMean "Signal"Std Deviation "Noise"Signal/Noise
Simulated – mid213.01.7123.5
Acquired – mid206.31.8115.5
Simulated – low
Acquired – low65.60.9668.3

It seems counter intuitive, but as the image gets darker, with fewer photons collected, the photon noise actually decreases, as seen in the above table. Even though it is common to describe darker images as “noisier,” the proper description is that darker images have a lower signal to noise ratio.

Both visually and through the statistical analysis of the images, there is good agreement between Imager and the image taken with the IDS camera with an Edmund lens.

Resolution Comparison at Best Focus On-Axis, Off-Axis, and with Defocus:

As we zoom into certain features of the images, we can see how well Imager models the characteristics of the camera system.
The acquired and simulated images below show that both on- and off-axis regions have similar resolution and focus clarity.

Best focus resolution

Fig 5: Details at best focus both on and off-axis


In the on-axis images, note that the vertical resolution fans on the right show similar resolution of between 6 and 7 line widths. Qualitatively, the blur and readability of the number labels on the resolution fans are identical, and the horizontal ticks next to the fan on the right look nearly identical, showing the effect of having a pixel resolution that is about 1/3 of the feature size. The impact of aliasing is noticeable in these images as well.

In the off-axis images, the resolution fans indicate a limit of about 4 line widths, and the legend numbers are less readable. The effects of distortion are clearly seen at the left and bottom edges of the images. The leftmost resolution fan and the bottom resolution fan are tilted away from vertical, – both effects result from the distortion of the lens.

A second set of simulated and acquired images were generated after defocusing the lens. The lens was rotated clockwise, bringing it closer to the sensor, and changing best focus from 0.9m to 5.25m. Looking at the same regions after adding defocus, the resolution achieved on the horizontal fan has dropped to 3 line widths in both images. The same regions of the simulation and acquired images show the expected loss in clarity and the benefit of using an accurate optical propagator in the Imager model.

Defocused resolution

Fig 6: Details away from best focus both on and off-axis



Overall there is good qualitative and quantitative agreement between Imager’s simulated images, and those acquired with the IDS uEye camera and Edmund Optics 8mm lens.  Stay tuned for future posts that will go into more detail about how this experiment was setup and comparisons of Imager’s output to the output of other hardware systems.

  • Ronald Zimmerman
    Posted at 14:21h, 13 April Reply

    Truly impressive qualitative argument for simulation.

    • Kenny Kubala
      Posted at 17:36h, 13 April Reply

      Thank you for the feedback. I appreciate it.

Post A Comment