David Bordwell's website on cinema   click for CV

Home

Blog

Books

On the History of Film Style pdf online

Reinventing Hollywood: How 1940s Filmmakers Changed Movie Storytelling

Film Art: An Introduction

Christopher Nolan: A Labyrinth of Linkages pdf online

Pandora’s Digital Box: Films, Files, and the Future of Movies pdf online

Planet Hong Kong, second edition pdf online

The Way Hollywood Tells It pdf online

Poetics of Cinema pdf online

Figures Traced In Light

Ozu and the Poetics of Cinema pdf online

Exporting Entertainment: America in the World Film Market 1907–1934 pdf online

Video

Hou Hsiao-hsien: A new video lecture!

CinemaScope: The Modern Miracle You See Without Glasses

How Motion Pictures Became the Movies

Constructive editing in Pickpocket: A video essay

Essays

Lessons with Bazin: Six Paths to a Poetics

A Celestial Cinémathèque? or, Film Archives and Me: A Semi-Personal History

Shklovsky and His “Monument to a Scientific Error”

Murder Culture: Adventures in 1940s Suspense

The Viewer’s Share: Models of Mind in Explaining Film

Common Sense + Film Theory = Common-Sense Film Theory?

Mad Detective: Doubling Down

The Classical Hollywood Cinema Twenty-Five Years Along

Nordisk and the Tableau Aesthetic

William Cameron Menzies: One Forceful, Impressive Idea

Another Shaw Production: Anamorphic Adventures in Hong Kong

Paolo Gioli’s Vertical Cinema

(Re)Discovering Charles Dekeukeleire

Doing Film History

The Hook: Scene Transitions in Classical Cinema

Anatomy of the Action Picture

Hearing Voices

Preface, Croatian edition, On the History of Film Style

Slavoj Žižek: Say Anything

Film and the Historical Return

Studying Cinema

Articles

Book Reports

Observations on film art

Archive for the 'Film theory: Cognitivism' Category

Protected: Poetics and interpretation

This content is password protected. To view it please enter your password below:

Protected: Studying the spectator

This content is password protected. To view it please enter your password below:

Cognitive film studies: The Hamburg Variations

Lobby, Von-Melle-Park 9, Universität Hamburg, with Murray Smith.

DB here:

How many universities in the US would post a big quotation from Karl Marx in the lobby of a  classroom building? (Answer: None.) But a cartoonish rendition of Marx’s killer app–the task of philosophy being not only to understand the world but also to change it–greeted the one hundred or so scholars attending the latest conference of the Society for Cognitive Studies of the Moving Image last week at the Universität Hamburg.

How much we’ll change the world remains to be seen. What is clear now is that thanks to the superb organization of Kathrin Fahlenbach, Maike S. Reinerth, and their team, the SCSMI had a lively four days of papers, discussions, and excursions–as well as a screening of a strong film by a skillful director previously unknown to me. In all, every good reason to go to a conference.

 

Big questions, candidate answers

Carl Plantinga, Dirk Eitzen, and Kathrin Fahlenbach on the Hamburg Harbor cruise.

The range was huge, as usual, because this is a very diverse group. Our community, as I’ve mentioned in earlier reports, hosts film scholars, scholars of other media (television, games, internet), psychologists, and philosophers. Recurring questions include: How do viewers comprehend media presentations? What is the role of morality in film and television? How do practitioners conceive their craft? How do media present and trigger emotions? What appeals are specific to certain genres or historical periods? Not least: How may we construct good theories to answer such questions?

Three sessions ran simultaneously at each time slot, so I missed many good presentations. Kristin and I occasionally doubled up. Many of the talks will eventually become published papers, several in the Society’s journal Projections, so let me just highlight a few strands.

For those of us interested in how style interacts with narrative, there were several significant presentations. James Cutting, who has pioneered the use of Big Data techniques in scanning large batches of films, provided a compact account of how “sequences” (as opposed to scenes) displayed patterns of coherence and grouping. Karen Pearlman discussed her films, and James examined them to reveal fractal patterns in their rhythms–patterns that match the dynamics of footsteps, heartbeats, and breathing.

Two papers added to our understanding of the Kuleshov effect, much discussed on this site. Marta Calbi and her collaborators conducted EEG testing on spectators exposed to a Kuleshovian film scene, while Anna Kolesnikov, another member of the team, traced the contradictory versions of the “experiments” that Kuleshov and his students purportedly constructed. Both were exciting contributions to our understanding of Kuleshov’s legacy, which as the years go by looks more complex than we had thought.

Other papers on visual style included Barbara Flückiger’s tour of the staggering resource she and her colleagues have devoted to the history of color in film–a database of images from 400 films illustrating a range of technologies, periods, and genres. She has also mounted a magnificent array of tools for detailed analysis of color. (Significantly, she has taken the images from film prints, not video copies.) Here’s one of several stunning images she has taken from Helmut Käutner’s Grosse Freiheit Nr. 7 (1944, Agfa nitrate).

Stephen Prince, whose book Digital Cinema has just been published, surveyed the emergence of “deep fakes,” manipulated images that have no basis in reality but which are indiscernible from an accurate image. Steve was once skeptical that digital images would ever cut us off entirely from photographic reality, but now machine learning has managed to do it. Sped up by social media, the rise of unverifiable “records” creates what he called “an attack on an information system we call society.”

Hitchcock, everybody’s idea of a shrewd filmmaker (when will cognitivists discover Lang?), came in for his usual close treatment. Todd Berliner proposed five ways in which story information can be set up and paid off in a narrative, and he used Psycho as an instance of at least four of those methods. Maria Belodubrovskaya countered Hitchcock’s constant claim that he relied on suspense and not surprise. She showed that both single sequences and overall plots (e.g., Suspicion, Psycho) were heavily dependent on surprise effects.

In the first day, the range of the “humanistic” side of SCSMI was on vivid display. Malcolm Turvey asked whether the give and take of philosophers’ arguments converged on a consensus about truth, the way that much of science progresses. To illustrate the process, he considered the return of an “illusion” theory of images put forth by Robert Hopkins. Later that day, Jeff Smith used the cognitive theory of confirmation bias to show how characters in Spike Lee’s Clockers systematically misunderstand the story situation. The result, enhanced by genre conventions and a violation of some traditions of the detective story, enmeshes the viewer in the same error.

On the other end of the spectrum, there were papers mobilizing empirical research. Timothy Justus drew upon Lisa Feldman’s work in How Emotions Are Made to suggest that a biocultural account of facial expression should not rely too strongly on Paul Ekman’s classic argument about universal, basic expressions.  More room, Justus proposed, should be given to cultural forces that build “emotion concepts” that filmmakers and viewers draw on.

A more hands-on empirical project was that coming from a team at Aarhus university. Jens Kjeldgaard-Christiansen posed the question: What if sympathy for a film’s villain varies with the personality of the spectator? As he put it, “If you are like a villain, will you like a villain?” So via Amazon’s Mechanical Turk the team ran questionnaires testing how personality types matched tastes in villains. Not to spoil their public announcement, I’ll just say that the results seem to confirm the belief that the most dangerous creatures on earth are young human males.

There was a lot more here than I can report, including provocative keynote addresses from Professors Anne Bartsch and Cornelia Müller (presenting work done with Hermann Kappelhoff). There was also a moving tribute to Torben Grodal, a central force in SCSMI and a leading-edge theorist applying evolutionary psychology and embodiment theory to traditional genre studies.

And, since SCSMI wants to understand how creators create, there was a film and its maker.

 

Without bullshit

In the past, our conferences have featured Lars von Trier and Béla Tarr, talking with us about their work. This time the guest was Thomas Arslan, a prominent “Berlin-school” director, who brought his 2010 film In the Shadows (Im Schatten) to the lovely Cinema Metropolis.

The film is a dry, hard-edged crime exercise. Trojan, fresh out of prison, comes to his boss to claim his share of the loot. In a tense confrontation he manages to get some of what he’s owed, but he’s now a target for elimination. He searches for a new robbery scheme and settles on the heist of an armored car. As usual, things don’t go as planned, particularly due to the intervention of a crooked cop.

Shot in a laconic style, In the Shadows took advantage of the Red One camera to produce remarkable depth in dark surroundings. (We saw it as a print because in 2010 theatres weren’t quite ready to show digital files. The print looked fine.) The drab locations complemented the anti-psychological impact of the story. Characters barely speak, the music is spare and almost subliminal, and the framing doesn’t accentuate the action expressively. In his post-film discussion, Arslan explained that the style grew out of the small budget and the rapid schedule. The result was a sort of modern B-film, avoiding the excess of CinemaScope and explosive chase sequences.

Arslan refuses to glamorize his gangsters, but he summons up sympathy for Trojan by minimal means. The protagonist is a solid, efficient professional, and he’s loyal to his boss until he realizes he’ll be betrayed. And as Hollywood screenwriters would remind us, you can sympathize with someone who’s been treated unfairly. I liked the unfussy handling of the story; it was a bit reminiscent of Rudolf Thome’s bare-bones Detective (1968). Arslan would be a good director to shoot a Richard Stark noir. “He does it without bullshit,” Arslan said of his thieving protagonist. It’s a good description of Arslan’s own technique.

 

SCSMI continues its tradition of merging humanistic inquiry with findings from empirical science. We can fruitfully ask questions that cut across traditional boundaries if we ground those questions in clear concepts and solid evidence. Our gathering next year moves back to the States–Grand Rapids, no less. More fun on the way.


Go here for earlier years’ coverage of SCSMI conferences.

On Richard Stark noir novels, see here and here. In the Shadows would fit nicely into my discussion of conventions of the heist film.

My own SCSMI contributions were two: a tribute to Torben Grodal and a little talk on contemporary women’s domestic thrillers (Gone Girl, The Girl on the Train, etc.). I may post the latter as a video lecture later this summer.

James Cutting lecturing at the SCSMI conference.

You and me and every frog we know

Frog tv 500

Frog TV (2014, N. Joe Myers).

DB here:

Be patient, the frogs are coming. But first, a bit of film theory.

 

Movies in code

Froggy 1 400

Once many film scholars were captivated by the idea that our responses to a movie were coded, we might say, all the way down.

Here, code means something like a way of assigning meaning that is both arbitrary and relative to time and place. The code is arbitrary in that it might just as easily have been otherwise. A code is relative because in different circumstances it might vary.

One prototype for the early semiologists was the code regulating traffic. In a traffic light, there is no natural or necessary connection between green and “Go!” or red and “Stop!” We’ve just agreed on these meanings. We could have assigned red to “Go!” and green to “Stop!” And indeed other cultures might do just that. A gesture may mean something friendly in one culture and something naughty in another.

The presumption is that because a code is both arbitrary and relative, it has to be learned. You need to live in a culture to understand the traffic code and the code of gestures. Through either observation or training (such as learning the codes of written language) you master the codes of your culture.

Clearly, movies include many codes. A film can show us traffic lights or hand gestures. Films include language as well, which is coded to a high degree. The crucial question is: How much in our responses to film is coded?

Almost everything, some said. There was a wing of semiology ca. 1970 that suggested that both what’s represented in film and the ways things are represented are arbitrary and culturally variable in the extreme. Some theorists suggested that just recognizing an object in a shot relies not on natural perception but rather on a code. Indeed, the argument was made that “natural” perception is itself coded. A favorite corollary of this was that language was such a powerful code that it reshaped perception. Put bluntly, if your culture’s language doesn’t distinguish red from green, you won’t see the traffic light’s top and bottom lamps as different colors.

 

I resemble that remark; or do I?

Froggy 2 400

Surely, somebody will say, many film images resemble the things they portray. A shot of a man and a woman looks, in certain relevant respects, like a man and a woman. We’re able to recognize them as such. Yet 70s semiology held that there was nothing natural about this. The image-object resemblance, often called “analogy,” and “iconicity,” was considered to be coded.

Here, for instance, is Umberto Eco in 1967:

We know that it is necessary to be trained to recognize the photographic image. . . . Even if there is a causal link with the real phenomena, the graphic images formed can be considered as wholly arbitrary.

Christian Metz brings out the cultural variability of recognition in a 1970 essay:

The apprehension of a resemblance implies an entire construction whose modalities vary notably down through history, or from one society to another. In this sense the analogy is, itself, codified.

Here is Dudley Andrew commenting on this line of thought in 1984:

The discovery that resemblance is coded and therefore learned was a tremendous and hard-won victory for semiotics over those upholding a notion of naive perception in cinema.

The semiological tradition pointed out, correctly I believe, that the parallel between the image and the thing it represents lies not in some relation between the two but in the comparable ways in which we understand each one. But can this process be considered both arbitrary and relative?

Can we, for instance, imagine a culture in which recognition was fundamentally arbitrary—where a picture of a bunch of bananas depicts, by cultural fiat, a clutch of cherries? More exactly, where a picture of a bunch of bananas is seen as a clutch of cherries?

We can imagine a very capricious filmmaker who set up a code with an introductory credit:

Every time I show you a shot of bananas, see cherries.

But you couldn’t see cherries. At best, the result would involve imagination, not perception. Even cooperative spectators would recognize the bananas as bananas, though they would go on to think of cherries.

The distinction between perception and thought seems to have been blurred in semiological theory. Perhaps the 70s arguments traded on the various meanings of “perception”, such as “I perceive that he’s unhappy today” or “I want to change your perception of Rosicrucianism.” The same would go for various senses of “recognition”: “I recognize the father’s face in the son,” “He had aged so much I didn’t recognize him.” These senses of the word bring in matters of conception and attitude, not just visual input.

 

Perception and evolution

Froggy 3 400

Not that perception is easy to define. When the semiologists were writing, they may have been influenced by psychological theories that assigned a big role to concepts in perception—“top-down” models that claimed perception to be “cognitively penetrable.” It’s fair to say that there’s now good evidence that the perceptual activities involved in visual recognition are largely fast, automatic, fairly dumb, and cognitively impenetrable.

Another error, it seems to me, was also the product of the period: the ignoral of evolution. The 1970s advocates of sociobiology seem to have been simply ignored, perhaps because they seemed to be naïvely attempting to see culture as nature. Yet I think that J. J. Gibson was right (also in the 1970s) in proposing that evolutionary theories could help explain why perceptual recognition is so swift and efficient.

Creatures evolved in an environment that selected for accurate perception, including very very very fast recognition of predators, prey, and mates. Evolution, we might say, provided long-term survival training in recognition, weeding out any creatures who didn’t register salient features of their world. A hominid who needed explicit teaching to recognize a tiger wouldn’t last as long as one who came to the task with senses pre-tuned. As with learning spoken language, all that would be needed is some exposure to the regularities of the ancestral environment. The baby born into our world is bombarded with redundant information that is non-arbitrary and non-relative; she quickly grasps that a face is more informative than a knee. It will take her quite a bit more time, along with some tutelage, to learn traffic signals and even more to learn to read.

Cinema taps some of our most fundamental capacities. Film images aren’t reality, but they present certain salient triggers for us to recognize what they represent. Indeed, the very perception of motion in movies is an illusion—a glitch in our visual system that cannot be arbitrary or culturally relative. Nobody in any time or place sees movies for what they are, a succession of still images. There are enough cues for movement to fool everybody’s visual system. And films preserve enough other features of the world to trigger automatic recognition processes. Perceptual recognition of something in the image is on the whole neither arbitrary nor culturally variable.

The likeliest conclusion is that we share with other creatures some native propensities which, given the right early exposure to the physical and social world, will be exercised in comparable ways. Of course there are differences. Minnows can’t enjoy Matisse, and we can’t smell as well as dogs can. Still, sensory systems overlap among many species.

So consider the frogs of the field: they toil not, neither do they spin.

 

Frogs like the front row too

We know quite a bit about how frogs see the world. A classic paper in animal neurology, “What the Frog’s Eye Tells the Frog’s Brain,” was crucial in suggesting the highly specialized nature of nerve fibers feeding information to the visual system. Those data streams—about edges, movement, and variable illumination—get reintegrated, but not necessarily thanks to high-level mental activity. “Early vision,” in frogs and in us, involves not some some brainy entity interpreting the image on the retina but rather several specialized, quite stupid systems that pool information before thought ever enters the picture. No codes necessary. What came to be called “distributed processing” does a large share of the job.

Though untrained in the codes of our culture, these frogs appear to recognize what matters to them, even on the mini-movie screen. (Sorry the video embed no longer works.)

If pigeons can quickly learn to interpret human faces in pictures, and female jumping spiders react to video images of males as if they were real, we shouldn’t be surprised that frogs can respond very—ah—actively on first exposure to movies showing things they really care about.

 

Of course a lot of our activity isn’t as data-driven and mandatory as recognition. Higher-level concepts do all sorts of work when we watch movies, from understanding who the protagonist is to grasping a film’s theme or point. And of course all manner of conventions in films require sophisticated cultural knowledge. My point is just that some aspects of our response are very basic. I hazard that our responses to cinema mix together all kinds of skills, abilities, biases, and proclivities. A movie is a package of appeals, a buffet with something for all our visual and auditory appetites.


First, thanks must go to N. Joe Myers, who posted this neat experiment in natural perception a year ago. (Has the filmmaker tried it with an iPad or a computer monitor that would make the worms look huge? Would that induce large-scale frog panic?) For vastly more frog-related information, go to N. Joe’s page here.

Thanks also to Darlene Bordwell, who called the piece to my attention, and Barb Anderson who pointed me to the spider studies. The illustrations are of course from Chuck Jones’ Warner Bros. cartoon One Froggy Evening (1955).

My quotations are from Umberto Eco, “Articulations of the Cinematic Code,” and Christian Metz, “On the Notion of Cinematographic Language,” both in in Movies and Methods, ed. Bill Nichols (University of California Press, 1976), 594 and 584; and Dudley Andrew, Concepts in Film Theory (Oxford University Press, 1984), 25. See also Metz, “Au-delà de l’analogie, l’image (1969),” Essais sur la signification au cinéma, vol. II (Klincksieck, 1972): “Resemblance itself is codified, because it makes appeal to a judgment of resemblance: the judgment of an image’s degree of resemblance will vary according to the time and place” (154).

J. J. Gibson’s writings mounted a strong case for what he called “direct perception” in The Ecological Approach to Visual Perception (Houghton Mifflin, 1979). His ideas were developed by Claire F. Michaels and Claudia Carello in Direct Perception (Prentice-Hall, 1981) and, more critically, by James Cutting in Perception with an Eye for Motion (MIT Press, 1986); pdf. Joseph Anderson’s The Reality of Illusion: An Ecological Approach to Cognitive Film Theory (University of Southern Illinois Press, 1998) is the trailblazing application of Gibson’s ideas to cinema. More generally, Paul Messaris’ Visual Literacy: Image, Mind and Reality (Westview, 1994) reviews the empirical findings that show virtually no cultural variability in perceiving line drawings and photographic images as depictions of recognizable persons, places, and things.

At some point in discussions of the cultural variability of perception, someone is likely to mention Eskimo cultures’ purported range of words for snow. But this is a long-standing error. See Geoffrey K. Pullum’s The Great Eskimo Vocabulary Hoax and Other Irreverent Essays on the Study of Language (University of Chicago Press, 1991) and John H. McWhorter’s The Language Hoax: Why the World Looks the Same in Any Language (Oxford, 2014).

What happened to these 1970s beliefs about the film image’s coded nature? It seems to me that when semiology as a research perspective waned, academics mostly stopped talking about the issue of resemblance and recognition. Researchers retained a general sense that films are coded, but they concentrated on less controversial conventions, like social stereotyping, representations of power, and so on. I’ve been told that the level of perception is merely a matter of “physiology” (wrong word) and thus of no consequence for the important issues that scholars should be studying. But of course filmmakers shape our response chiefly by controlling our perception, and if we want a comprehensive account of how films work and work on us, we can’t ignore this level of the viewer’s activity. I consider this problem in the title essay of Poetics of Cinema.

For a more developed argument about how films bundle a range of appeals, from sensory triggors to high-level concepts, see my essay, “Convention, Construction, and Cinematic Vision” in Poetics of Cinema. Online, I discuss these matters here and here and here (under “Representational Relativism,” which involves not frogs but our ancient ancestors).

P.S. 21 September 2015: Apes appear to remember episodes after watching movies. Thanks to Bill Evans.

Froggy 4 500

One Froggy Evening.

David Bordwell
top of page

have comments about the state of this website? go here