A digital image of a painting is not the painting. A digital image of a folio is not the folio. It is an artifact in its own right that conveys, or fails to convey, the information one would want to study. Just as human perception is full of complexities, the construction of a digital facsimile is full of complexities. A very simple image may be recognizable as a representation of the original, while a very advanced image may fail to satisfy the questions of research. The project of “digitization” requires reflection on the nature of scholarly perception of the artifact and the digital tools suited to capture and represent the pertinent information. It is no longer enough to consider spatial resolution (pixels per inch or centimeter). This presentation considers first the modes of perception in scholarly investigation of artifacts such as manuscripts including the importance of texture and interactivity for humanities questions. Second, we will consider the ability of “spectral imaging” (including narrow-band reflectance, fluorescence, and transmission) to meet and surpass the capabilities of the human eye on first-hand inspection. Third, we will consider the tools for capturing and representing texture and interactivity with raking light photography and Reflectance Transformation Imaging (RTI). Finally, we will present the results of a recent project, funded by the U.S. National Endowment for the Humanities, to integrate Spectral Imaging and RTI (Spectral RTI) in capture, processing, and visualization. The software and documentation created by the project are freely available online and accessible for use “off the shelf” by imaging teams.
Scholarly investigation utilizes a variety of modes of perception depending on the questions brought to the artifact. Often the interest is in reading text written on the artifact, which can be difficult if the manuscript was deliberately erased for reuse or otherwise damaged. The difficulty only increases if the writing was secondary in the first place, as with marginal or interlinear notes or quire numbers. Some scribal markings, such as scoring lines, are not even inked in the first place. Similarly, dry-point notation could be an important object of study, yet completely invisible in diffuse-light photography. Limited considerations for digitization may be appropriate if a manuscript is valued only as a text container. Different considerations for digitization are appropriate if a manuscript is valued as an artifact of scribal cultures. To all these examples from manuscripts many more could be added for other media, such as paintings. The study of such objects in person, without digital or other mediation, involves first and foremost movement. The scholar moves the relationship between the eye, the object, and the light, and changes the lighting. The movement could mean moving closer or farther, to a different angle, or even holding the object up to a light to see if light passes through. Just as no one still position satisfies every inquiry, no one still photograph creates an adequate facsimile for scholarly investigation. Indeed, the experience of interaction surpasses the sum of individual moments of perception. The value of an interactive experience is strong for research, and even stronger for teaching. For all these reasons it is still common to hear “there is no substitute for first-hand experience.” While experience with a digital surrogate may never be identical, the number of questions that can be answered from the surrogate depends on the methods of digitization. The limits of digitization are not absolute. When justified, the use of spectral and texture imaging may allow a digital copy to approach and in some ways surpass first-hand experience.
Spectral imaging expands the range and resolution of color perception of the human eye. The human range spans from violet to red, excluding ultraviolet on one end and infrared on the other. Human color resolution is limited to three receptors, so all the colors we see are combinations of intensity of three elemental colors, red, green, and blue. Digital spectral imaging measures reflectance of pixels in an image at specific wavelengths of light. These wavelengths can extend to ultraviolet and infrared, and resolve colors more finely than the three receptors of the human eye. Just as someone with only two kinds of receptors can be called “color blind,” all the more so a person with three receptors is limited compared to the sixteen or more possible with spectral imaging. In addition to reflectance, spectral imaging today often captures fluorescence and transmission, comparable to use of a blacklight and holding a light behind the object, respectively. Because the capture is digital, the numerical values of light intensity at each picture element under each of dozens of conditions can be processed in relationship to each other. As a result, color can be captured and rendered more accurately than a simple “color” camera, and algorithms can enhance contrasts, such as erased ink on parchment, that could never be perceived with the unaided eye.
Texture imaging is most simply accomplished with raking light. That is, the light is positioned from one direction at a low angle to the object. Any readable image of a coin, inscription, or cuneiform tablet uses this approach. The main limitation of raking light imaging is the judgment of the photographer in anticipating all the light positions that would be necessary to interpret the artifact. One solution is capture a complete set of raking light images from forty to sixty different positions. Again, since the information is digital and can be easily processed by algorithms, such a data set could be processed into a dynamically relightable image. This technology, called Reflectance Transformation Imaging (RTI), can extrapolate to light positions not directly captured, can enhance the texture, and is fully interactive. Imaging the texture of a surface is distinct from, but complementary to, 3D imaging. 3D imaging, using techniques such as laser scanning or photogrammetry, digitizes the boundary structure of an object. Texture imaging digitizes the reflectance properties of the surface as a function of light position, usually at much higer resolution. 3D imaging is well suited to capturing the shape of an object. For any one surface on that shape, RTI is well suited to capturing the fine texture, roughness, and specularity (shininess). The two technologies are entirely compatible, as a 3D engine can render both the shape of many surfaces (represented as triangles) and the shader properties of each surface.
Digital spectral imaging and RTI have been in development since the beginning of the century when high-quality digital imaging became widely available. The recent development is the publication of the technique, open-source software, and documentation for integrating spectral and reflectance transformation imaging (Spectral RTI). A pair of grants from the U.S. National Endowment for the Humanities supported the experiments and development of the technique, followed by full-scale implementation, documentation, and open-source software that performs the processing entirely within a graphical user interface. The fundamental premise of the integration is that chrominance and luminance can be combined in color spaces such as YCbCr. Spectral imaging is concerned with chrominance and strives to avoid highlights and shadows by using diffuse illumination. RTI is concerned with luminance variation as a function of light position (i.e., highlights and shadows generated by light from specific angles) and pays no more attention to color than what is captured from a conventional camera with a Bayer filter. All the data can be captured with one camera and combined in post-capture processing. All the required software is freely available, most notably the SpectralRTI_Toolkit plugin for ImageJ.
Further reading