Ione, A. (1999). Defining visual representation as a creative and interactive modality. In R. Paton & I. Neilson (Eds.), Visual Representations and Interpretations (in press). London: Verlag-Springer Publishing Co.. Presented at Visual Representations and Interpretations’98. 22nd - 24th of September, 1998, The Foresight Centre, University of Liverpool, London, UK

 

Defining Visual Representation as a
Creative and Interactive Modality

Amy Ione
PO Box 12748
Berkeley, California 94712-3748
ione@lmi.net

 

1 Introduction

 

Images are everywhere today and when we look at them closely it quickly becomes clear that the technological innovations of the twentieth century have allowed us to see formerly unavailable information. One compelling aspect of this, and my primary area of concern in this paper, is that while artists have pierced through surfaces and created dynamic pictures of the brain working, scientific imaging technologies have generated images that have revolutionized our understanding of our minds. This paper discusses these new ways of 'seeing' the world and ourselves, defining visual representation as a creative and interactive modality.

 

 2 Ways of Seeing Images

  

People often define images in terms of iconography, meaning the images offer symbolic or descriptive illustrations of a subject. This kind of definition mitigates the element of communication that is a part of image-making and a part of the relationship we establish with an image when we look at it. The film maker Sergei Eisenstein clearly described this interactive form of creativity when he wrote:

In fact, every spectator, in correspondence with his individuality, in his own way and out of his own experience . . . creates an image in accordance with the representational guidance suggested by the author leading him to understanding and experience of the author's theme. This is the same image that was planned and created by the author, but this image is at the same time created by the spectator himself. [1, p. 33; 2, p. 261]

Let me propose that even adding this interactive element to our perception of an image still fails to adequately address how images have enlarged our body of knowledge, our sense of who we are, and have also expanded our visual range. Non-optical images show how this expansion is possible. Thus, they offer good examples of creativity as an interactive modality, as well as good examples of how images have helped create our consciousness and our cultural environment.

More specifically, and turning to scientific discovery first, in 1873 Sir John Eric Ericksen, a British surgeon appointed Surgeon-Extraordinary to Queen Victoria, said "The abdomen, the chest, and the brain will forever be shut from the intrusion of the wise and humane surgeon." These words clearly did not leave room for the discovery of non-optical imaging in 1895, two years later. This discovery made it possible to see inside our bodies without cutting and before physical disintegration of the flesh.

The non-optical revolution per se could be said to have begun when Wilhelm Conrad Roentgen discovered that radiation can penetrate solid, opaque substances like human skin and that it has the same effect on a photographic plate as light. Roentgen called this discovery the X-ray, having no idea what it was. Today, the event is generally characterized by the skeletal image of a ringed X-ray hand.

This image was not trivial. The exceptional element relevant to my thesis that images offer an interactive modality - a modality that can expand human understanding beyond known parameters -becomes evident when we look at the electromagnetic spectrum. The electromagnetic spectrum has a visible portion that includes wavelengths ranging from red light (at about 700 nanometers) down to violet light (at about 400 nanometers). This is the section where we find all of the colors of the rainbow. The self-propagating waves that comprise the entire spectrum include a variety of electric and magnetic fields that like visible rays travel at the speed of light. These invisible waves, however, have different frequencies from visible light and it is frequency that determines whether we characterize waves as radio, microwave, infrared, visible light, ultraviolet, X-rays, or gamma rays.

Looking at the spectrum of visible and invisible waves together underlines that while we can not pierce through opaque surfaces like skin with our eyes or even with photography per se, we can do so with invisible radiation. We can also use this radiation to create maps, and since Roentgen's time, many non-optical inventions have been developed to help us map and explore areas formerly unknown to the human mind. The key point here is that the discovery of the X-ray is what made this investigation into domains invisible to the eye possible. Again, it was because he found himself face-to-face with the bones within his hand that Roentgen investigated. This investigation of the anomaly, in turn, is why Roentgen became the first person to realize how the use of invisible radiation can produce and record a visible image of an invisible object property. [3]. In sum, his inquiry offers an example of how mind and phenomena can come together to define a scientific problem that formerly did not exist.

 

3 Images and Creative Interaction

The investigation of the image, however, ultimately highlights areas that stretch far beyond the initial X-ray image. Thus these areas are critical in considering visual representation and interpretation, especially from a cognitive perspective. For example, one intriguing element we find here is that Roentgen's discovery was a non-algorithmic insight. This means that the insight that opened a new domain scientifically, medically, culturally, and philosophically cannot be classified as something that followed a logical or strictly empirical pattern. On the one hand, this was because the image revealed something that was not directly perceived - since both the radiation and the human bones are invisible to the unaided human eye. On the other hand, the image was not directly received. In other words, the event was not an insight alone or a revelation he intuited indirectly. Rather, the interaction between the mind and the physical world led to the insight. This means it is essentially incomplete to characterize the event as something that was analytically, intuitively, or psychically conceptualized. Moreover, it cannot be emphasized enough that there was no mathematical correlate initially. This means that all people, including Roentgen grasped what they saw with their eyes and also immediately conceptualized that other possibilities existed. This led some to then develop the dependable technologies that are now capable of making the skin as well as other physical and opaque surfaces transparent.

These technologies have now become so familiar that people overlook the degree to which the initial discovery altered the way people experience the world. For example, while scientists like Roentgen were enthusiastic about creating extensions that would allow them to open additional doors to formerly unavailable information, some lay-people like Frau Roentgen had little enthusiasm for the new information being revealed. After confronting her skeletal hand in her husband's laboratory Frau Roentgen was convinced it was an omen of death and never returned. As events like the nuclear disaster at Chernobyl show, Frau Roentgen sensed the negative side of radiation that has slowly revealed its face to us.

There were also people, like the Russian painter Pavel Tchelitchew, who saw the X-rays as a door to another dimension. He creativity recorded his perception and because he documented it we can now see it in his painting Hide-and-Seek, a work produced in 1942. Tchelitchew's painting quickly authenticates itself as a twentieth century artifact, for in looking at the image closely one sees it displays a chorus of X-ray images of children's see-through heads arranged in puzzle-like patterns as parts of a growing tree. The veins and bones of the children merge with the roots and bark of the tree in a now-you-see-it, now-you-don't puzzle pattern of arteries and landscape [4]. The piece also conveys the interior of the bodies and includes an accurate physiological rendering of skeletal structure as well as the kinds of internal elements that became transparent only with the imaging technologies that developed as a result of the X-ray image and knowledge of radiation waves. [4].

It should be noted that what is not included in the image also identifies Hide-and-Seek as a piece of the twentieth century. For example, history shows the idea of rays piercing the body was not in and of itself totally novel to the twentieth century. To the contrary, ideas of rays had been around for centuries. Even as far back as the thirteenth century the philosopher Roger Bacon had noted that no substrate is so dense that it can prevent rays from passing through and Bacon pointed out that the walls of a vessel of gold or brass show this when they heat up. [4]. There was also the belief that spiritual rays emanated from the body to the outside world, and these were often portrayed in paintings by halos around the heads of saints and religious figures.

Yet, and this is a key point, these earlier ideas were not actually the ideas validated by imaging science. Visual representations of earlier artists and philosophers did not suggest that any rays, whether passing from the inside or emerging from the soul out through the skin, could reveal everything beneath the skin to a human eye - the way a non-optical image does. There was also no suggestion that the images could leave an impression on something else, like a shadow on a wall or a permanent imprint on glass or film. Yet radiation rays were soon tamed to do so.

This is not to say that art and artistic representations did not continue to inform the human dialogue. For example, Gerald D. Fischbach, a Professor of Neurobiology, described Hide-and-Seek as a painting that

[C]aptures the interplay between the mind and environment that influences the brain's development as well as its architecture. Hidden forms are embedded figures, a delicate test of mental function. Roots, branches and vines suggest neuronal arborization and the ability of such structures to change. [5, p. 49]

On reviewing Fishbach's description in relation to the discovery of non-optical imaging one is inclined to ask why cognitive scientists have mainly devoted their efforts to the study of laboratory problems. Short laboratory exercises cannot address or replicate the complexity of a work of art or the kind of creative activity that Roentgen's discovery reveals. ## ( In Roentgen's case, as explained earlier, problem-solving could not even begin until he actually recognized that there was a problem that needed to be solved. Until the image was visually present to Roentgen, there was no 'problem' to solve. It was simply understood that we cannot see through our skin. This kind of unknown problem is the kind of creative experience that can not be tested for in a lab.

 

The larger point is that this kind of creative and interactive modality is hard to derive experimentally and often becomes mentally transparent to us once a new discovery is actively being incorporated into scientific work and our cultural experience. ** (For example, to continue to build on the case study the X-ray provides, once discovery of non-optical image formation made it possible to see beyond the visible as it had been previously defined, the technology was quickly combined with computers. This combination generated an avalanche of data-driven inventions, all instrumental in expanding how we know the physical, the invisible, who we are, and how our bodies function. Now when reviewing the assortment of simulations, photographs, and digital renderings that are used to describe our world, our bodies, and our ideas, it is difficult to categorize what is virtual, what is actually seen, and what is better described as a representation of something outside of our visual reach.

Perhaps it is this knowing that there is always information beyond our reach that has led some, like Semir Zeki, a Professor of Neurobiology and Co-head of the Wellcome Department of Cognitive Neurology at University College London, to ask if art offers an uncharted area. In a recent article called "Art and the Brain," Zeki correctly notes that while a great deal has been written about the visual brain, little has been written in relation to one of its major products, art. [8, p. 71]. Zeki then writes:

I hold the somewhat unusual view that artists are neurologists, studying the brain with techniques that are unique to them and reaching interesting but unspecified conclusions about the organization of the brain. Or, rather, that they are exploiting the characteristics of the parallel processing-perceptual systems of the brain to create their works . . . [8, p. 77]

This intriguing statement is worth exploring in light of how cognitive neuroscience uses brain imaging methodology and how the imaging technology is providing a biological link to complex tasks such as visual attentional control, memory storage, language interpretation, and brain functions. The resulting maps illustrate human coritical processing with millimeter and millisecond resolution and follow chemical processes in normals. Recent technological advances have also made it feasible to create pictures of our brains as we think, learn, read, and visualize. Images of these kinds of processes are generally formed over time by an orderly set of operations that include, for example, placing the parts of the images in their proper relationship and scanning the content for specific features [9]. As a result, and for the first time, scientists are able to render certain aspects of thought visible by recording the physical effects of brain activity. Overall, the images generate descriptions of mental processes, show active and dormant areas of the brain, and show that component operations can be precisely and visually specified. The excitement surrounding this new information attests to the undeniable ways in which visual representations have extended our understanding of brain functionality. The images do not, however, provide the kind of methodological reach that necessarily acknowledges images as a creative and interactive modality.

 

4 Areas for Future Investigation

The failure to see how the development of non-optical images added a new perspective is an area that has not yet been effectively been addressed in cognitive neuroscience research. Given this, I would like to propose that the relationship among the following four elements is of primary importance to a comprehensive understanding of cognitive involvement with visual representations. I would also like to propose that these four areas should be given more attention as a whole -especially if we are to forge a more comprehensive understanding of what images are and how new discoveries inform both science and culture.

First, as outlined earlier, the discovery of x-rays made it possible for researchers to see a domain not accounted for in earlier philosophical, religious, artistic, and scientific investigations. Thus it raises the question of whether it is possible to design an experiment that is in fact capable of mapping, cataloging, and characterizing visual and logical possibilities not yet imagined in any fashion. Given the scope of this question, I will only state here that research by myself [10-13] and others [14-16] shows the answer is no. In another context, the Nobel Prize winner Leon Lederman recently explained the problem many are pushing aside when he said: "There's always a place at the edge of our knowledge, when what's beyond is unimaginable, and that edge, of course, moves. . ." [17]

Second, and on the counter-side, the x-rays led to the discovery of the various data-driven non-optical technologies researchers now depend on to map and study the brain. This use of new technologies to map and monitor our minds indicates that once we have developed new means to access data and engage in intelligent problem-solving we can creatively extend how we know and use available information. This is not research into a new and unknown domain, but a form of discovery that consolidates information using comparative and correlative forms. It is creative to some degree but, again, not a way of addressing 'unexpected' discoveries so much as a way that extends ideas easily characterized as within the range of problems researchers are already actively pursuing.

Third, cognitive science literature and the literature on consciousness has tended to either ignore creativity or to assume that problem-solving and personal insight are the only viable options we can apply to creative developments. The limitations within this perception are readily apparent when we look at how creative work develops, be it in art, science, or elsewhere. This is an especially important point given that many in psychology, philosophy, the history of science, and art history - who do not adopt the cognitive science approach - have shown that the stress on universals favored by cognitive scientists cannot address the complexity of individuals, the nature of creative products, and the dynamics of a living and evolving creative process. [14; 16]. Moreover, as has been documented, even cognitive scientists who are known for their interest in creativity, like Newell and Simon [18-20], do not even have entries for creativity or even creative thinking in their indexes. Thus, despite the fact that they propose their scientific work in problem-solving is relevant to creative thinking, whatever their understanding of creativity is remains implicit to them, rather than expressed.

Finally, it is not just our experiments that must be looked at closely. We must also look at the way in which human minds and cultural ideas work in tandem, a point David Teplica, a Chicago based photographer and plastic surgeon, alludes to by incorporating radiographs into his 1989 "Birth of Man with Homage to Michelangelo."

Teplica gives the impression that he has simply X-rayed Michelangelo's original image, a kind of stylistic allusion that acknowledges art of the past while taking it out of context to be used in a new way. [4, p. 283] Michelangelo would never have been able to conceptualize why Teplica is using X-rays in his replication of God touching Adam, or even explain what the radiographs are. An even larger, by still related point, and one I have expanded on in previous work [10; 11; 21], is that implicit prejudice often engages our minds and these biases cannot be logically refuted since they are mentally invisible and impossible to physically model and study in a lab.

 

5 Conclusion

In summary, visual representations offer a means to consider how problem-finding, problem-solving, iconography, and image development differ and interpenetrate. As I have demonstrated, scientists are using some measure of artistry to evolve digital image formation, interpretation, and clarity. Artists, on the other hand, have used digital advancements as well as traditional artmaking techniques to personalize the domain scientific technologies has revealed. The sum total of this is that we have become a more visual culture, a culture that has shown visual representations are a creative and interactive modality.

References

1. Eisenstein, S. Film sense. Harcourt, Brace & World New York, 1947

2. Gruber, H. E., & Davis, S. N. In: R. J. Sternberg (ed.) Inching our way up Mount Olympus: the evolving-systems approach to creative thinking. The nature of creativity: contemporary psychological perspectives, Cambridge University Press Cambridge,1988, 243-270

3. Beck, R. N. In: T. Umiker-Sebeok (ed.) The Future of Imaging Science. Advances in Visual Semiotics -- The Semiotic Web Walter de Gruyter, Mouton Publications, Berline, 1994, 609-642

4. Kevles, B. H. Naked to the bone: medical imaging in the twentieth century. Rutgers University Press New Brunswick, New Jersey, 1997

5. Fischbach, G. D. Mind and brain. Scientific American, September, 1992; 2673: 48-57

6. Holland, J. H. Emergence: from chaos to order. Addison-Wesley: A Helix Book Reading, Mass, 1998

7. Scott, A. Stairway to the mind: the controversial new science of consciousness. Springer-Verlag New York, 1995

8. Zeki, S. Art and the brain, Daedalus, 1998; 1272: 71-104

9. Posner, M. I., & Raichle, M. E. Images of mind. Scientific American Library New York, 1997.

10. Ione, A. Information: Description, Cognition, Invention, Informatica, 1997; 213: 421-434

11. Ione, A. An investigation into emergence consciousness, Performance Practice, 1998; 3:16-18

12. Ione, A. "Science: method, myth, metaphor?" Alexandria, 5:1999, in press

13. Ione, A. Multiple Discovery, Encyclopedia of Creativity Academic Press San Diego, 1999, in press.

14. Gardner, H. Extraordinary minds. Basic Books New York, 1997

15. Csikszentmihalyi, M. Creativity: flow and the psychology of discovery and invention. HarperCollins New York, 1996

16. Sternberg, R. J. (ed.). The nature of creativity: contemporary psychological perspectives. Cambridge University Press, Cambridge, 1988

17. Dreifus, C. Science is serious business to the 'Mel Brooks of physics'. New York Times, July 14, 1998

18. Simon, H. A. Models of my life. Basic Books New York, 1991

19. Newell, A., Simon, H. A. In: J. Haurgeland (ed.) Mind Design. Computer science as empirical inquiry: symbols and search. A Bradford Book, Cambridge and London, 1981, 35-66

20. Newell, A., Simon, H. A. Human problem solving. Prentice-Hall Englewood Cliffs, NJ, 1972.

21. Ione, A. Implicit cognition in scientific speculation and development. Paper presented at the first conference of the Association for the Scientific Study of Consciousness, Claremont Colleges and Claremont Graduate School, Claremont, California, June 14-16, 1997

 

 

[##] Gestalt images are often used to represent brain-mapping. Like the brain-mapping itself, however, a gestalt image fails to represent the kind of discovery that raises the two-sided whole to another level. Emergence is the term generally used to explain this kind of conceptual development and how this cognitive development is a part of parallel processing. While the concept emergence seems to infer development, actually explaining what it is has proved to be difficult. Of more concern is that the growing acceptance of the idea of emergence has led people to use the term as if it stands for something 'real,' despite the fact that it is only a theoretical idea that resolves something we know happens that essentially continues to be a mystery. The critical point here is that, on the one hand, the idea does fill the definitional vacuum. On the other hand, the growing acceptance of the idea brings to mind that when Newton introduced the idea of gravity he specifically stated that he did not know what gravity is. He was using the word to explain something evident in his mathematical equations. As history shows, eventually people became so accustomed to the idea that gravity was a 'real' force that no one looked further, at least not until Einstein's relativity challenged some basic assumptions within the Newtonian system. Two books that attempt to make sense of the problems evident in the idea of emergence are Alywin Scott's Stairway to the mind: the controversial new science of consciousness and John Holland's Emergence: From Chaos to Order [6; 7]

[**]Even if we use the Gestalt idea that the whole is more than (different from) the sum of the parts the Gestalt interpretations can only account for the horizontal mental exchange. Saying the patterning and relatedness between the parts and a whole is more than (different from) the sum of its parts is not represented. It is a conceptual idea. Thus the image does not explicitly convey how an emergent leap beyond all that is presently known among us comes to be a part of our knowledge base as a whole. In other words, we know the image and its parts differently after the cognitive leap that allowed us to see the ambiguous reading is possible. Moreover, each time this kind of cognitive leap occurs we are reminded that there is a clear and distinct difference between an 'old' way and a 'new' way of 'seeing.' The 'new' way of seeing was simply not possible within the 'old' framework. It must be emphasized that the history of creativity and discovery shows that each time we see anew, we are once again reminded that all conclusions are drafts subject to change.'