INTRODUCTION TO THE SERIES
FocalPoints® constitutes a treatise on the modern imaging system. The acronym, FOCAL, defines the five essential parts of the imaging task. Field describes the real-world scene that will be captured, Optics transfers information about the scene to the sensor, Conversion represents the detection of photons and the conversion to a digitized signal, Algorithms explores the wide range of signal processing, and finally Limn, an uncommon but apt term for the visual depiction of information, addresses the presentation and observation of visual information.
ELEMENTS OF THE DIGITAL IMAGING SYSTEM
In this article, we discuss the five fundamental elements of the modern imaging system. At FiveFocal, we view imaging as a complete system that begins with the object and ends with the observer. Seen in this way, the design of an imaging system requires knowledge of each of the elements in the chain and optimal transfer of information from one element to the next. We have created internal tools, methods and design procedures that take all of the imaging system components into account for all of our projects.
Field
The properties of the object, or scene, under observation set the
fundamental requirements of the imaging system. The object consists of a time-changing three dimensional volume with varying absorption, transmission and reflection spectra. The object is observed by the measurement of its electromagnetic field, which may be self-generated (e.g. in the case of fluorescence or thermal IR) or a function of the object’s illumination.
The imaging system can use illumination that varies as a function of intensity, space, time, spectrum, coherence and/or polarization to help better understand the object. Augmentation with active illumination gives additional information when compared to using only the scene’s natural illumination and can be used to adapt to transient events, for example, moving objects or atmospheric turbulence.
Some examples of systems that heavily use the illumination degree of freedom to enhance information capture are LIDAR, barcode scanners and strobes for capturing images of objects moving at high speeds.
Optics
In the most general sense, the optics or lens is an angularly selective volume which allows the measurement of the electromagnetic field from distinct points in the field of view. The optic’s primary purpose is to undo the effects of propagation from the object to the lens. Propagation from a point source, as was described by Huygens in the 1600’s, effectively spreads the field equally into a spherical wavefront. The optics job is to capture the field and to reverse the phase of the spherical carrier wave so the field may be concentrated for detection.
The optics consists of a volume of reflective, refractive and/or absorptive material that can change as a function of time. When considering this general definition, standard refractive lenses, grin lenses, zoom lenses, mirrors, pinhole cameras, holograms, and lens-less imaging systems are all included. All of these systems can collect information beyond just the DC value of the scene and can be used as part of an imaging system. The appropriate optical volume must be designed to provide adequate signal at the detector, adequate dynamic or static field of view, the ability to image the object space volume of interest either through variable focus or depth of focus, the minimization of stray light effects, the ability to provide the required resolution over the specified wavelength range, polarization and coherence and are bound by cost.
In practice, the optical volume has a transfer function, called the optical transfer function (OTF) that describes how well the system produces information at the image plane and includes the effects of optical aberrations. This transfer function is a low pass function, due to the effects of diffraction and aberrations, which limits the information from the scene that can be relayed to the detector.
Conversion
The conversion function, or sensor, consists of a single, array or volume of photosensitiv
e regions. The photosensitive regions can have varying geometries, spectral and polarization sensitivities, gains and efficiencies. The goal of the photosensitive regions is to capture the angularly and/or spatially separated information that is presented by the optics. Once the electromagnetic field has been mapped from the object to the sensor, the photons are converted into charge and then digital signal. The photosensitive regions can sample the incoming field at different wavelengths and polarizations, with different start, end times and can be read out either in parallel or serially. The sensor’s role in the imaging system is to make the photon to electron conversion efficient, again preserving key information about the object.
As was true with the optics, the sensor also has a transfer function which is determined by its geometry and its effective pixel size. In many systems it is the sensor, and not the optics or object characteristics that limit the information that is captured. Examples of components that perform conversion are CCD and CMOS sensor arrays.
Algorithms
The algorithms/signal processing receives the digitized data and either attempts to make the best image for human observation or to accomplish a specific task. The signal processing has a transfer function of its own and has the similar characteristic that it is limited by the information transferred to it from the previous component. Information cannot be created by this component; at best it can only be preserved and in some cases emphasized. Examples of algorithms that condition raw visible spectrum color data for human observation include demosaicing, pixel defect correction, auto white balance, color correction, auto exposure, auto gain control, lens shading, gamma, noise 
reduction, sharpening and compression. Examples of algorithms that are task based are barcode readers, iris identification, pedestrian detection, lane departure detection, etc. These algorithms do not provide any imagery; they simply give the user heavily compressed information. Instead of an image of an iris, they tell the user whose iris it is, for example.
The signal processing transfer function can be extremely complex as the algorithms can modify their behavior based on the signal itself (both spatially and temporally), the environment, the user interface, availability to the network/cloud, and external sensors such as light and motion sensors. The implementation of algorithms is a function of the hardware the algorithms will run on and it requires power, logic and memory which leave the designer with a complex trade space to navigate.
Limn
Limn (display) defines what is done with the output of the algorithm. It includes displaying captured information for visual observation, sending visual information to a network, or sending actionable information based on the results of a task based algorithm. 
In the case of displaying visual information, the design of the complete imaging system includes specification of the display contrast, brightness, and resolution sufficient to provide adequate viewing at a reasonable distance. If the imaging system is to work with a given display, it is critical its effects be included in the system specifications. For example, the color saturation and low spatial frequency contrast of an image displayed on a small cell phone screen is much more critical to human observation than high frequency information which cannot be presented at that level. If the image is expected to be magnified on the device or presented on a larger display, high frequency information becomes critical, as do artifacts that are frequently introduced by compression algorithms. As was true with the sensor, the transfer function of the display is a function of it geometry, color fidelity and dynamic range.
Summary
In all our projects, we consider the requirements, components and the interplay of the transfer functions of all aspects of the imaging system. While a straightforward design for a camera objective lens, for example, may be performed without consideration of the other components, the best performing, and most cost effective imaging systems are developed with an eye to creating subsystems that work optimally well together.