Discovery Imaging and Reimagining Discovery

1. Introduction

It is an honor to be with you today sharing ideas about discovery, cultural heritage, and technology. I’m particularly excited about the potential for sharing ideas between people working on similar goals in the United States and in Italy. I'm very grateful that you allow me to speak to you in my native tongue. Se provo a pronunciare una frase in italiano, sarai grato se continuo in inglese. In other ways, I am eager to learn about how questions that interest me may seem different from your context.

2. Columbus

As I think about discovery here in northern Italy, I can’t help but think of a famous discoverer from Genoa. I hope you will be patient with me for a minute as I follow a tangent about Christopher Columbus. It will bring me back to my main point, that when we talk about discovery we should think about who is discovering what for whom. Growing up in the United States, I was taught that Christopher Columbus discovered America. As my education advanced and my culture shifted, that statement took on new meanings. Today, I see that statement not as about history from 1492, but about history from the nineteenth, twentieth, and twenty-first centuries in the United States. Wikipedia hosts a list of Christopher Columbus statues around the world.¹ There are a small number on various islands in the Caribbean. The nation with the third most monuments is Spain. The nation with the second most monuments is Italy. The nation with the most monuments by far is the United States. Even accounting for the liklihood that contributors to English Wikipedia have interests and sources biased toward the United States, I think a basic conclusion is inevitable. The United States is obsessed with valorizing the discovery of America more than any other nation in the world. Why is that? What is really at stake? What do we mean by discovery? Who discovered what for whom?

There are at least two answers from different points in the history of the United States. This statue titled “The Discovery of America” was on the steps of my nation’s capitol from 1844 until 1958. The United States in the 1840s was not concerned with the history of Italy, Spain, or the Caribbean. The statue was a proxy for the discovery of what is now the western United States. Thomas Jefferson purchased the Louisiana Territory from Napoleon and sent Lewis and Clark to explore a route from the Mississippi River to the Pacific Ocean. Lewis and Clark discovered many resources, most notably land suitable for agriculture not yet broken up into individually owned tracts. The only impediment was an ecosystem of buffalo and semi-nomadic people following them. The statue is not about discovery in the sense of making visible what had not been visible before. It was about the superiority of an economic system based on individual property ownership, and the valor of expanding that economic system. Western expansion, what we called “Manifest Destiny,” was presented as the latest in a history of glorious triumphs. Christopher Columbus became a type or model for a generalizale principle of the conquest of inferior civilizations by superior civilizations. Who discovered what for whom? Anglo-Americans from the eastern United States discovered potential for economic expansion for Anglo-Americans from the eastern United States.

By the dawn of the twentieth century, a different question took priority. Who is a real American? Our Civil War and abolition of slavery led to a constitutional ammendment that defined anyone born in our territory as a citizen. My point for now is not about the formerly enslaved, but about migrants not born in our territory. Our industry benefitted greatly from cheap labor, but accepting migrants or their children as real Americans was a different question. We developed the category of “white” to distinguish real Americans from our guest workers. Migrants coming from southern Italy and Sicily were not allowed to check the box for “white” when they arrived at our ports. Italian Americans thought history was on their side. Christopher Columbus was Italian. An Italian discovered America. Italian Americans are more American than the English or any other latecomers. Columbus Day first became a holiday in my country as an attempt to appease the Italian government after eleven Italian immigrants were lynched in Louisiana. Discovery is about claiming rights. Who discovered what for whom? An Italian discovered America as a home for his his fellow Italians, and Italian Americans discovered a claim to to equal status as Americans. ²

Now in the twenty-first century, the conversation is about who is excluded when rights are claimed. There is yet another interpretation of the historical data and a new critique of earlier narratives. There seems to be a hope that discovering historical accuracy will expand equal rights. Christopher Columbus did not discover America. Amerigo Vespucci was a different person. There were already people there when he arrived. America and the United States are not the same thing. Christopher Columbus was not a nice person. We still want a day off, but let’s call it Indigenous Peoples Day rather than Columbus Day. Christopher Columbus is talked about in the United States as much as ever. The discovery that Columbus did not discover America is itself treated as a discovery. As someone trained in the history of biblical interpretation, I fully expect that another interpretation will be discovered in the historical data at some point in the future. Historical data requires interpretation. As always, discovery is about something larger than uncovering hidden things. No interpretation or narrative is final. What we count as a discovery is subject to interpretation by future generations.

3. For Whom

So now you might understand why my interest in discovery includes examining our assumptions about who is discovering what for whom. What does digital technology allow us to do that could not be done before? Certainly one important kind of discovery occurs when a scholar who already has access to a manuscript is able to read a letter that was not legible without digital technology. I would like to turn to the discovery made by someone who has never touched parchment, who discovers a manuscript by way of a screen. This profile describes the vast majority of my students. For a wide variety of reasons, most of them will never come to Milan. They will never have the privilege of access to the reading room of the Biblioteca Ambrosiana. Are they capable of experiencing discovery? My answer has to be yes. So my question becomes, what technology is necessary to facilitate their discovery?

For me, the Biblioteca Ambrosiana is a place of discovery. I have been privileged enough to study Latin Moses, including Latin Jubilees and the Testament of Moses in person. I have some digital images of that manuscript that I would like to show on screen for your consideration.

4. Texture and Interactivity

The image on the left is a high-quality calibrated color image of a page of the manuscript. Most digitization projects would call a collection digitized if they had images as good as the image on the left for every page. Indeed, it is a great image and I would not want to do without it. Yet, if I walked into this room right now and you told me, “Quick! Point to Latin Moses!” I would point to the image on the right. That is what the pages look like in the reading room of the Biblioteca Ambrosiana. I’m not just talking about conservation status and scribal features such as ruling lines. I’m talking about the experience of holding something from the fifth century that has passed through many hands since. The image on the left misrepresents the manuscript as an artifact of scribal culture. If a manuscript is only a text container, I might prefer the image on the left. Even more so, I would prefer an enhanced pseudocolor image or even a scan of the editio princeps, which is far more readable than any of the alternative images. Discovery in the context of material philology cannot be content with the standard of legibility of text.

As I reflect on my own experience with studying manuscripts and the experiences I wish to share with my students, there is another aspect that is not captured even in the image on the right. I think about examining manuscripts behind glass, and how much worse that is. Even then, I would move my head around to catch better glimpses of the manuscript. Remove the glass and my response is not simply to get closer, but to interact with the manuscript. Of the technologies that bring interactivity to digital images, perhaps the simplest is Reflectance Transformation Imaging, or RTI. RTI allows a user to move a virtual light around the object to visualize the texture through highlights and shadows.

As I have shared digital facsimilies with my students, the experience of interaction was a profound discovery moment, far more so than the legibility enhancements we also worked on. We are getting ever closer to engaging with manuscripts and other artifacts in virtual reality, augmented reality, and 3D immersive environments. The gaming industry is driving the innovation on which progress for cultural heritage applications depends. We must be careful, however, to avoid the temptation of using generic libraries for texture, rather than captured texture.

5. Color

If texture is the aspect of material philology least appreciated, color is the aspect most falsely taken for granted. I have heard some Christians argue that the complexity of the human eye proves the existence of an intelligent designer, God. If that were the only evidence, I would conclude that God’s chosen creature is the octopus. The human visual system has real limitations, and much of what we perceive would be best characterized as hallucination. The problem is not just the color that we cannot see. Spectral imaging can help with the range and resolution of our color perception. The bigger problem is the color that we think is real but is not. If I showed you only the image on the left you would think it was a good image. It was taken with a high quality DSLR camera by a competent photographer. Only by comparing the two images side by side would the shortcomings of the image on the left become apparent. Depending on your background, it may not even be obvious to you that the image on the right is better. You might think the image on the left is prettier, or richer, or more colorful in a good way. The image on the right is carefully calibrated based on ten spectral bands. It is objectively better by the metric of accurate reproduction. The image on the left was captured by a camera designed according to the principle that if it looks good it is good.

But what is so bad about an image that looks better than accurate? Again, if the manuscript is only a text container, it may not matter. However, if we are going to start characterizing inks and pigments, color accuracy is a very big deal. If we are going to assess conservation status over time, color accuracy is a very big deal. If we are going to unify digitally folios from a manuscript that was divided into different collections, color accuracy is a big deal. If we are going to study material philology such as the methods of parchment preparation, color accuracy is a big deal. The privilege of first-hand access to the original artifact is no longer an excuse to neglect recording data that will be of utmost importance to scholars in other parts of the world and in the future.

6. Accessibility

My final point on reimagining discovery and our work creating digital images for discovery brings me back to the question of discovery for whom. Who has access to the data? Are the restrictions on access based on measured judgment, or simply lack of follow-through? What are our obligations to others who may not know or follow our way of doing things? Questions of access have long been fundamental to those working in library sciences. The latest acronym responding to the proliferation of big data, especially in the sciences is FAIR. FAIR stands for Findable, Accessible, Interoperable, and Reusable.

This graphic shows that permission is only one of the many considerations that can prevent discovery. Much of it has to do with machine readability and machine actionability. Turning specifically to the work of images of cultural heritage artifacts, especially manuscripts, we should discuss IIIF, TEI, and Creative Commons.

The International Image Interoperability Framework (IIIF) checks most of the boxes for the FAIR principles. The framework includes APIs that allow images and regions of images to be shared at the needed scale or in tiles. It allows groups of images to be richly descibed with all the metadata that one would need to navigate in the Mirador Viewer. Mirador viewer is important for multispectral imaging because it allows many registered images to represent a single page or canvas. The capabilities for annotation, authentication, and search are quite impressive. All the metadata is machine actionable.

In my opinion, the limitation of IIIF is that the standards excel at machine readability more than human readability. It can be tedious for a human to enter all the seemingly redundant data that the IIIF APIs require. For that reason, I am an advocate of using TEI as the foundational collection of metadata, and deriving the IIIF format of metadata from TEI. The Text Encoding Initiative is often thought about as a way of encoding transcription. It is really much more than that. It can record everything that a IIIF Presentation Manifest would need. It can describe the facsimile images and regions thereof. It can hold the metadata about a manuscript. Some use TEI even when there is no text transcription because of the header and rich potential for recording metadata.

Since I criticized the IIIF APIs for their human readability, it is reasonable to wonder how TEI compares in this regard. I have been playing with angle brackets since I first learned about this new thing called the World Wide Web back in 1993. I count it as a strength that a closing tag is reasonably descriptive, compared to the anonymous closing brackets in IIIF JSON. I can understand how someone would find the tags disruptive to human reading, but I think syntax highlighting can fix that. Syntax validators can usually guide us directly to where we might have missed a closing tag.

If one can agree that XML is a reasonable balance of human and machine readable, the remaining question is TEI. I once had a collaborator decide that he should print the complete guidelines and take them home for a little light reading. I feel bad about the trees that had to die to complete that print job. Admittedly, the guildelines can be overwheling if one tries to take them all in at once. However, major sections can be skipped entirely by someone like myself. I never paid attention to the codes for poetry, theater, speech transcription, and notated music. For the sections that do interest me, such as manuscript description and critical apparatus, the detail is most welcome. I find it telling that TEI has so few tutorials not produced by the TEI Consortium. The official guidelines provide sufficient simplicity to get started with basic examples, and as much complexity as is necessary for one digging deeper with a particular problem. The fact that TEI allows us to include all the information we would want for cataloging, access, and visualization brings some complexity, but an elegant complexity. If one embraces the concept of semantic encoding and the value of making assumptions explicit, the exercise of encoding text can itself lead one to discovery.

The final impediment to discovery that I would like to discuss today is permission and license. Some rights holders still seem to fear that the value of the original object will decrease if digital facsimiles are freely available. I do not pay to hear a live performance or see important works of art because I have not heard and seen digital facsimiles. On the contrary, the digital preview is what motivates me to pay for the non-digital experience. A more subtle problem is mistaking putting something on the internet for providing access. The internet is dense with humans and machines, and some will take whatever they can find. More reputable aggregators, however, may not index a resource if permission to do so is not explicity given. Permisison has to be explicit and machine actionable. Using a standard license and encoding it in a machine-actionable way provides the best path to discovery by my students, who invariably rely on machines to guide their exploration of their curiosity.

7. Conclusion

In conclusion, I encourage those of us working in cultural heritage to think more deeply about who is discovering what for whom. Columbus may not have been the first person to journey from Europe or the Mediterranean to the western hemisphere. If he had not made it back and opened a path to others, I do not think we would be talking about him today. If we capture data or process an image, but do not open a path for others, should we really call that discovery? It is still covered for most.

We should also think about what we are discovering. It makes a difference whether Columbus himself discovered a shorter route to the indies, or discovered that the earth is spherical, or discovered a new hemisphere of land, cultures, and crops, or somehow spriritually if not literally founded the imperial spirit of the United States. For us, it makes a difference whether our discovery work is focused on manuscripts as text containers, or manuscripts as artifacts scribal cultures to be studied with material philology.

Similarly, it is worth considering the danger of misrepresenting what we find. The problem with the statue on the steps of my nation’s capitol was not that it celebrated the encounter between cultures, but that it misrepresented the cultures. The tremendous advances in availability of digital images of manuscripts have led many scholars to think or even say that they have examined a manuscript when they have only examined digital facsimiles of that manuscript. I recently read some scholars boasting of a “digital twin” for its close resemblance of the original, and I think they were even implying equivalence. This is a problem that needs to be addressed on many fronts. I very much agree that such scholars need to be educated with appropriate humility. I also understand that in many circumstances, calibrated color and texture imaging may not be feasible. I would argue, however, that if scholars see enough high-quality digital representations of color and texture, they will naturally become skeptical or humble about what claims can be made based on simple “better-than-nothing” digital images. Past generations have illustrated the dangers of short-sighted and selfish pursuits of discovery. The application of chemical reagent to manuscripts reflects the subconscious notion that my generation is the pinnacle of technology, and I represent my generation. We may not be harming manuscripts in the same way, but “better-than-nothing” imaging could still do damage if it leads to the false impression that advanced imaging is not necessary. Natural forces may damage the manuscript before future generations have the opportunity to rectify our short-sightedness. Some of the earliest photographs of the Dead Sea Scrolls are still the most valuable not because of the photographic quality but because of the moment in time which they captured. The various temptations that keep us from preserving and sharing what we call discovery today will harm future generations who will invariably wish to reinterpret our discovery and make discovery of their own.

Thank you very much.