William P. Seeley

There has been considerable interest in recent years in whether, and if so to what degree, research in neuroscience can contribute to philosophical studies of mind, epistemology, language, and art. This interest has manifested itself in a range of research in the philosophy of music, dance, and visual art that draws on results from studies in neuropsychology and cognitive neuroscience.[1] There has been a concurrent movement within empirical aesthetics that has produced a growing body of research in the cognitive neuroscience of art.[2] However, there has been very little collaboration between philosophy and neuroscience of art. This is in part due, to be frank, to a culture of mutual distrust. Philosophers of art have been generally skeptical about the utility of empirical results to their research and vocally dismissive of the value of what has come to be called neuroaesthetics. Our counterparts in the behavioral sciences have been, in turn, skeptical about the utility of stubborn philosophical skepticism. Of course attitudes change… and who has the time to hold a grudge? So in what follows I would like to draw attention to two questions requisite for a rapprochement between philosophy of art and neuroscience. First, what is the cognitive neuroscience of art? And second, why should any of us (in philosophy at least) care?

There are obvious answers to each of these questions. The cognitive neuroscience of art is a subdivision of empirical aesthetics devoted to just that, the application of neuroscientific methods to the study of our engagement with artworks (more on the cognitive bit later). We should we care? Neuroscience helps us sort out the kinds of information processing involved in our psychological engagement with the world. So neuroscience is germane to the task of evaluating whether philosophical theories about our engagement with art reflect our best understanding of the psychological processes that underwrite them. But, of course, this claim is really just a hackneyed naturalistic platitude. And platitudes too often leave too many stones unturned to be of much use. The devil is always in the details. In this case the devil is a question of pragmatics, or a question about the real methodological utility of neuroscientific research to aestheticians and philosophers of art in particular cases. So the obvious answers turn out not be so easy.

I am not sure there was a neuroscience of art a decade or so ago. There is a branch of experimental psychology called empirical aesthetics. This field traces its roots back to a book published in 1871 by Gustav Fechner called, of all things, On Experimental Aesthetics. Fechner was a key figure in the development of the new field of psychology in the nineteenth century (he was instrumental in the development of psychophysics). So empirical aesthetics is as old as psychology itself. This should come as no surprise. Baumgarten introduced the term “aesthetics” in the eighteenth century to refer to a science of sensuous cognition. Nonetheless, a decade ago the idea of a genuine experimental neuroscience of art was only just emerging as a productive possibility. The literature consisted largely of perspectives pieces drawing connections between results in neurophysiology, facts about the formal structures of particular artworks, and anecdotal stories about the productive practices of particular artists.[3] This literature pointed towards the promise of a neuroscience of art. But it was missing the marks of a true experimental science: empirically testable hypotheses and associated experimental research.[4] This is changing.

A general model for a cognitive neuroscience of art has emerged from this early literature.[5] Artists develop general formal vocabularies and particular compositional strategies via a systematic exploration of the behavioral effects of different sets of marks, movements, tones, rhythmic patterns, or narrative devices. We need not overplay the use of the term ‘systematic’ in this context. The process need not be explicit. The claim is simple and pragmatic: formal strategies develop relative to their success or failure as a means to evoke desired behavioral responses in consumers. This suggests a means to evaluate artworks as a class of stimuli. Cognitive science, in its most general sense, is the study of the ways organisms acquire, represent, manipulate, and use information in the production of behavior, or to coin an awkward acronym, ARMUI. Artworks are stimuli intentionally designed to induce a range of affective, perceptual, and cognitive responses in readers, spectators, viewers, and listeners. This suggests that we can model our engagement with artworks as an information processing problem: how do consumers acquire, represent, and manipulate information carried in the formal structure of these stimuli, and what is the relationship between these processes and those explicit behaviors associated with our canonically artistic engagement with this range of artifacts? Cognitive neuroscience is a tool that can be used to model these processes and behaviors. These models can in turn be used to evaluate alternative hypotheses about the nature of our engagement with artworks in a range of media. The answers to these kinds of questions can be used to gain traction in debates about the nature of art more generally. Therefore cognitive neuroscience is a tool that can be productively used to explore questions about the nature of art and aesthetic experience.

Why a cognitive neuroscience of art? I am often surprised by the degree to which the folks I interact with on the neuroscience side of these endeavors are committed to a core aestheticism. In this regard the term ‘neuroaesthetics’ isn’t just a name. It reflects an ideological bias about the nature of art. And this is a sticking point. I take it that issues germane to theories in aesthetics and the philosophy art can be peeled apart. There are questions about the aesthetics of nature, industrial design, graphic design, or etc. that are not artistic questions. There are questions about the meanings of artworks and the nature of our engagement with characters that are not aesthetic questions. More importantly, the philosophy of art encompasses questions concerning artistically salient aesthetic phenomena, but aesthetics does not encompass non-aesthetic semantic or ontological questions about the nature of art or our engagement with artworks. Therefore, not only are these two sets of concerns distinct, but the philosophy of art represents a broader view of art than aesthetics. Likewise, biased competition models for selective attention demonstrate a close connection between the meaning, identity, or semantic salience we attribute to a stimulus and the affective and perceptual features constitutive of our phenomenal experience of it. Cognitive neuroscientists use fronto-parietal attentional networks (feedback loops) that connect prefrontal areas (areas associated with object identification, working memory, and the attribution of affective salience to a stimulus) to sensory processing in the visual, auditory, and somatosensory systems to model these effects.[6] This suggests that the answers to questions about the semantic salience of artworks generally, issues that are central to the philosophy of art, play a regulative role at a neurophysiological level in determining the aesthetic quality of our engagement with particular artworks. Therefore a cognitive neuroscience of art represents a broader view of art than neuroaesthetics. So, what’s in a name…? The change I have proposed is an attempt to realign the research program within neuroscience in order to bring it into register with a more realistic view of the range of issues pertinent to the study of art.

Of course it is one thing to have a general, abstract model for the potential contribution of neuroscience to philosophy of art. It is another thing to have a good set of case studies that show that the model works passably well in a dirty, noisy, uncooperative environment. And this is where the pesky, persistent, nagging question, “Why should we care?” becomes important. For a long time the received dogma in computational theories of mind was that neuroscience is implementation-level science. Questions about the nature of a target behavior, what a system is doing, how does it represent information, etc., could be answered through functional level analysis. Neuroscience might tell us how these representations and processes were realized in a type of organism. But this, it was thought, wouldn’t contribute much to our understanding of its psychological behavior. This may not always be the case. The scenario I am envisaging is one in which a range of mutually inconsistent alternative theories are each consistent with the observable aspects of some target behavior. If evidence from neuroscience can provide some traction in our understanding the way a system in fact acquires, represents, manipulates or uses information in the production of the target behavior, then neuroscience contributes something novel to our understanding of what the system is doing, or the nature of the target behavior. The result need not necessarily favor one alternative over another. We might instead be forced to reconsider the distinctions that differentiate the alternatives. The canonical case study for this kind of claim in cognitive science is the imagery debate where, dogged disagreements about format aside, evidence from neuroscience demonstrates that modality specific imagery and perception share modality specific processing resources.[7] I have argued that the debate between Simulation and Theory-Theory approaches to narrative understanding provides an analogous example in philosophy of art.[8]

So one reason we should care is that neuroscience can contribute helpful information to entrenched philosophical debates. However, the utility of neuroscience to the philosophy of art does not hinge on the success of its application in controversial case studies. It is sufficient that neuroscience can help us gain traction in understanding the way artworks work, e.g., how they carry and convey their content. For instance, Noël Carroll has argued that part of the power of movies lies in their capacity to direct attention and frame the way we conceptualize and experience film narratives. In particular, he argues that filmmakers use various in-camera effects and editing techniques to focus viewer attention on particular aspects of scenes diagnostic for a directed interpretation of the narrative. These features determine the salience of current actions and events, foreshadow future actions and events, color our retroactive interpretation of previously depicted actions and events, shape our moral expectations about the unfolding lives of characters, and thereby drive our understanding and appreciation of movies. Mark Rollins argues analogously that paintings are perceptual stimuli intentionally designed to direct the attention of viewers toward their aesthetically and semantically salient features. Rollins argues that these strategies work by virtue of the fact that artists’ formal and compositional strategies tend naturally to become tuned to the operations of perceptual systems over time. This model can be generalized to other media. In this regard, artworks can generally be interpreted as exogenous, or externally imposed, attentional routines that carry the intentions of the artist. Carroll and Rollins thereby treat artworks as attentional strategies.[9] I propose that we shift the burden of responsibility away from the artist to the artwork in these contexts (in part to allow for contextual variance and avoid murky philosophical questions about the role of artists intentions in interpretation) and call them attentional engines, or stimuli designed to independently induce a range of experiences in consumers.

Research by Uri Hasson and his colleagues supports this general view of artworks.[10] There is a methodological problem that is a sticking point for any rapprochement between philosophy of art and neuroscience. Our engagement with artworks, like natural vision more generally, is messy. It doesn’t reduce neatly to the kinds of contexts that yield successful neuroimaging experiments. In a standard imaging study one systematically varies the value of one aspect of a stimulus, e.g., the brightness of a color patch. This yields carefully controlled data about change in underlying neurophysiological processes that enables researches to make inferences about discrete aspects of information processing in the brain. The trouble is that this method is poorly suited to spatiotemporally complex, dynamic stimuli whose content is constrained by a range of ill-defined contextual features, (e.g., film, dance, and natural vision). Hasson has developed a means to overcome this problem for natural vision using what he calls inter-subject correlation analysis (ISC). ISC is used to measure and compare the changing rate of activation over time in different brain regions among a range of participants who have been exposed to the same dynamic stimulus. Film and video are a means to present a repeatable dynamic scene to any number of participants. Therefore they are ideally suited stimuli for these experiments. Hasson has thereby winged two birds with one stone. He has developed a method for studying vision in (more) ecologically valid natural contexts that is also a valid method for a neuroscience of film.[11]

Hasson’s studies yield several types of results that support the interpretation of films as attentional engines. For instance, in one study participants were asked to lie on their backs in a scanner and watch the opening 30 minutes of The Good, The Bad, and the Ugly. The movie was presented on a computer screen and viewed in a mirror mounted over participants’ eyes. The sound track was provided via specialized headphones designed for use within the noisy, magnetized environment of the scanner. The instructions were simply to watch the movie. Participants were free to choose what to look at, how long to look at it, and etc. Despite the uncontrolled nature of the free viewing task there were high, statistically significant (p < 0.001) inter-subject correlations in visual areas involved in sensory processing, pattern, form, and face recognition, auditory areas (Herschel’s Gyrus), language areas (Wernicke’s Area), areas associated with emotional processing, and multisensory areas.[12] All in all ISC demonstrated time-locked processing among subjects in approximately 45% of cortex. These results contrast with results recorded from among groups of participants who were in complete darkness in the scanner and sets of participants who viewed different segments of the same movie. In neither case was there any evidence of ISC correlations. These results are interesting. However, they need not, in and of themselves, reveal anything significant about our engagement with movies. The trouble is the free viewing task. What one really needs is a way to analyze what participants are doing in order to confirm that the ISC measure reflects commonalities in the way participants attend to the film. This information emerges from two sources in Hasson’s research. Eyetracking data and gaze maps demonstrate that participants fixated their attention on the same locations at the same time while viewing the clip.[13] These results were replicated and extended in a separate study. Here Hasson compared ISC, eye movement, and gaze map data collected from a 10 minute clip of The Good the Bad, and the Ugly and a 10 minute, unstructured, one shot video of a people coming and going while listening to a Sunday morning concert in Washington Square Park in New York City. The unstructured real life event evoked far less ISC than the tightly edited film, particularly in areas beyond those associated with basic sensory processing.[14] Further, eye movements and gaze maps were closely correlated in responses to The Good the Bad and the Ugly, but in responses to the video of the unstructured real life event eye movements wandered and participants did not attend to the same locations.

So. There is a story about the cognitive neuroscience of art. There is a suggestion from within philosophy that movies are attentional engines, or that filmmakers have developed a set of techniques designed to capture and direct viewer attention to those affective and semantically salient aspects of scenes that carry critical information for the construction of film narratives. Hasson’s research lends support to this claim. I have focused on his work on visual attention in this discussion. These results generalize to ISC measures for the influence of auditory processing of soundtracks in our visual engagement with movies and are independently supported in research by Nicole Speer and her colleagues.[15] A biased competition model of selective attention can be used to model the associated behaviors.[16] In ordinary contexts, selection is a critical problem for perception. The environment is replete with information, only a small subset of which is salient in any given context. Add the fact that our basic processing resources are limited and we can readily see that we need a means to selectively filter information on the fly in order to efficiently collect the information necessary to achieve our immediate goals in real time. Biased competition models describe fronto-parietal attentional networks that direct eye movements, bias the sensitivity of populations of neurons in sensory cortices to goal related features of the environment, and thereby explain the influence of task relevance, semantic salience, and affective salience in perception and attention. These processes can, in turn, be used to model artworks in a range of other media as attentional engines.[17]

I suppose that in some sense none of this is a surprise. We perceive movies. One ought to, therefore, be able to model some aspects of our engagement with movies perceptually. It is likely true that this kind of claim generalizes to any of a range of non-art film and video stimuli, e.g. athletic contests and the nightly news. So, the question rises again…”Why should a philosopher care?” The short answer is that it gives us traction in understanding how artworks work. The longer answer is that an understanding of our engagement with artworks is important because, in the long run it should give us greater traction in a range of problems we are interested in. Is there a risk of default on this promissory note? I suppose. It is, after all, an empirical question how far this model generalizes to questions of interest to philosophers of art. However, artworks are cognitive stimuli. Therefore, whatever else we might think about issues of ontology or value, everything in the philosophy of art rides (I am willing to argue) on answers to questions about our engagement with actual artworks. These are by and large psychological questions about the ways we acquire, represent, manipulate, and use information in the production of behavior. Neuroscience is in the business of modeling answers to these kinds of questions. Where this can contribute information to help sort out difficult questions, resolve entrenched debates, or simply help confirm our best theories about the way artworks work neuroscience can make a productive contribution to philosophical practice. I’m willing to bet a couple (more) cases like this will emerge.



Notes

1. See J. Robinson, Deeper than Reason (New York: Oxford University Press, 2005); B. Montero, “Proprioception as an Aesthetic Sense,” The Journal of Aesthetics and Art Criticism, 64(2), 2006, pp. 231-242; and M. Rollins, “What Monet Meant: Intention and Attention in Understanding Art,” The Journal of Aesthetics and Art Criticism, 62(2), 2004, pp. 175-188.

2. See B. Calvo-Merino, D. E. Glaser, J. Grèzes, R. E. Passingham, and P. Haggard, “Action Observation and Acquired Motor Skills: an fMRI study with expert dancers,” Cerebral Cortex, 15(8), 2005, pp. 1243-1249; U. Hasson, Y. Nir, I. Levy, G. Fuhrmann, and R. Malach, “Intersubject Synchronization of Cortical Activity During Natural Vision,” Science 303, 2003, pp. 1634-1640; and M. Skov and O. Vartanian (eds.), Neuroaesthetics (Amityville, NY: Baywood Publishing Co., 2010).

3. See M. S. Livingstone, “Art, Illusion, and the Visual System,” Scientific American, 258(1), 1988, pp. 78-85.; M. S. Livingstone “Is It Warm? Is It Real? Or Just Low Spatial Frequency?” Science, 290, 2000, pp. 1299; S. Zeki and M. Lamb, “The Neurology of Kinetic Art,” Brain, 117 (3), 1994, pp. 607-636. See also M. S. Livingstone, Vision and Art: The Biology of Seeing (New York: Harry N Abrams, 2002); S. Zeki, Inner Vision (New York: Oxford University Press, 1999); and R. Gregory, J. Harris, P. Heard and D. Rose, eds., The Artful Eye, (New York: Oxford University Press, 1995).

4. A. Chatterjee, “Neuroaesthetics: A Coming of Age Story,” Journal of Cognitive Neuroscience, 23(10), 2010, pp. 53-62. The one domain for which this isn’t true is neuroscience of music which seems to emerge as a robust, coherent experimental discipline at about this time.

5. See N. Carroll, M. Moore, and W. P. Seeley, “The Philosophy of Art and Aesthetics, Psychology, and Neuroscience: Studies in Literature, Music, and Visual Arts,” in A. P. Shimamura and S. E. Palmer, eds., Aesthetic Science: Connecting Minds, Brains, and Experience (New York: Oxford University Press, 2011); W. P. Seeley “Cognitive Science and Art,” in S. Davies, K. Higgins, R. Hopkins, R. Stecker, & D. E. Cooper, eds., Blackwell Companion to Aesthetics, 2nd Edition (Malden, MA: Blackwell, 2009), pp. 191-194; W. P. Seeley and A. Kozbelt, “Art, Artists, and Perception: A Model for Premotor Contributions to Visual Analysis and Form Recognition,” Philosophical Psychology, 21(2), 2008, pp. 1-23; and Rollins, 2004.

6. See Seeley & Kozbelt, 2008; S. Duncan and L. F. Barrett, “Affect Is a Form of Cognition: A Neurobiological Analysis,” Cognition and Emotion, 21(6), 2007, pp. 1184-1211; S. Kastner, “Attentional Response Modulation in the Human Visual System,” in M. I. Posner, ed., Cognitive Neuroscience of Attention (New York: Guilford Press, 2004), pp. 144-156; L. Pessoa, S. Kastner, and L. G. Ungerleider, “Attentional Control of the Processing of Neutral and Emotional Stimuli,” Cognitive Brain Research 15, 2002, pp. 31-45.

7. S. M. Kosslyn, W. L. Thompson, and G. Ganis, The Case for Mental Imagery (New York: Oxford University Press, 2006); Z. W. Pylyshyn, Seeing and Visualizing: It’s Not What You Think (Cambridge, MA: MIT Press, 2003).

8. W. P. Seeley “Imagining Crawling Home: A Case Study in Cognitive Science and Aesthetics” (2010). Review of Philosophy and Psychology: Psychology and Experimental Philosophy, 1(3), 407-426; N. K. Speer, J. R. Reynolds, K. M. Swallow, and J. M. Zacks, “Reading stories activates neural representations of visual and motor experiences,” Psychological Science 20(8), 2009, pp. 989-999.

9. Rollins, 2004; Rollins, “Pictorial Strategies and Perceptual Content,” in H. Hecht, R. Schwartz, and M. Atherton, eds., Looking Into Pictures: An Interdisciplinary Approach to Pictorial Space (Cambridge, MA: MIT Press, 2003), pp. 99-112.

10. For a review of these studies see U. Hasson, O. Landesman, B. Knappmeyer, I. Vallines, N. Rubin, and D. J. Heeger, “Neurocinematics: The Neuroscience of Film,” Projections: The Journal for Movies and Mind, 2(1), Summer 2008, pp. 1-23.

11. See D. J. Levitin, This Is Your Brain on Music (New York: Dutton, 2006) for an analogous claim about audition and neuroscience of music.

12. See Hasson, 2008, figure 2.

13. See Hasson, 2008, figure 3.

14. See Hasson, 2008, figure 4.

15. Speer et al, 2009; Hasson et al, 2008. Hasson has also found systematic differences in ISC between different genres, e.g., a continuum from high to moderate ISC for Hitchcock suspense thrillers, Spaghetti Westerns, and contemporary sitcom comedies respectively.

16. See Seeley and Kozbelt, 2008; Rollins, 2004.

17. W. P. Seeley, “Seeing How Hard It Is: Selective Attention and Cross Modal Perception and the Arts,” unpublished manuscript presented at the Eastern Division Meeting of the American Philosophical Association, December, 2010.


2011 © William P. Seeley