{"id":37625,"date":"2017-09-03T15:15:01","date_gmt":"2017-09-03T20:15:01","guid":{"rendered":"http:\/\/www.davidbordwell.net\/blog\/?p=37625"},"modified":"2018-10-03T14:47:48","modified_gmt":"2018-10-03T19:47:48","slug":"venice-2017-sensory-saturday-or-what-puts-the-virtual-in-vr","status":"publish","type":"post","link":"https:\/\/www.davidbordwell.net\/blog\/2017\/09\/03\/venice-2017-sensory-saturday-or-what-puts-the-virtual-in-vr\/","title":{"rendered":"Venice 2017: Sensory Saturday; or, what puts the Virtual in VR?"},"content":{"rendered":"

\"Camera<\/a><\/p>\n

La Camera Insabbiata<\/strong> (2017) by Laurie Anderson and Hsin-chien Huang.<\/em><\/p>\n

DB here (still at the Mostra<\/a>):<\/p>\n

Kristin and I are Virtual Reality novices, so when we got a chance to sample some pieces at\u00a0Venice VR<\/a>, we jumped. The demand was great for the 22 pieces on display,\u00a0and you had to book each encounter separately in advance. The\u00a0event was held on its own island, Lazzaretto Vecchio<\/a>, which was startling\u00a0in itself; the facility was once a\u00a0hospital for plague victims.<\/p>\n

We took in four pieces in one afternoon and may go back to see some others. We didn’t see exactly the same ones, so my report and speculations are limited to mine. Apart from being fairly wild, sometimes creepy, sensory experiences, they set me thinking about how VR seems to work as a medium.<\/p>\n

 <\/p>\n

Angels of artifice<\/strong><\/p>\n

My quartet was a varied assortment, each somewhat parallel to a film genre or narrative prototype. The first was Greenland Melting<\/em>,<\/a> by Catherine Upin, Julia Cort, Nonny de la Pe\u00f1a, and Raney Aronson-Rath. It’s part of a Frontline\u00a0documentary series, and it follows the format, but with immersion. Presenters addressed\u00a0the camera (me) and took\u00a0me via helicopter to the glaciers so I could watch them heaving themselves into the sea.<\/p>\n

\"Greenland<\/p>\n

The camera took me underwater as well, and I experimented with voluntarily bobbing\u00a0under and above\u00a0the waterline; it worked. Again, as in a film, the transitions between shots was managed through quick dissolves.<\/p>\n

La Camera Isabbiata<\/em>, by Laurie Anderson<\/a> and Hsin-Chin Huang, is an installation running 20 minutes. It presents a fantasy landscape that you can move through. With two controllers, one in each hand, you choose one of several\u00a0virtual domains to explore. As far as I could tell, all of them were based on cavernous arrays of blackboards chalked\u00a0with graffiti. (See the image surmounting today’s entry.)<\/p>\n

I first chose the sound domain, which asked me to speak. My words were not only played back to me, but then assumed vase or candy-bowl shapes of gleaming color.<\/p>\n

\"Sound<\/p>\n

Getting bored with hearing myself and seeing my words made solid, I switched to flying.<\/p>\n

Flying was most excellent. Once I figured out how the remotes governed direction, speed, and angle, I could steer a swift path through, over, and around those huge black surfaces. The sense of self-initiated optical flow was powerful, although there were no other cues (e.g., balance, air pressure) to mimic actual flight. But I could have fun\u00a0playing with\u00a0this a long time. This was pure subjective\u00a0POV cinema, or first-person gameplay if you like, and everything was one single shot.<\/p>\n

If La Camera Isabbiata<\/em> put the onus on the user to initiate everything, Draw Me Close: A Memoir<\/a><\/em> by Jordan Tannahill, made me respond to a situation. Once the goggles went on, I was guided to a black-and-white broken-line image\u00a0of a woman on her sickbed. The artist’s voice-over presented the situation of his mother, dying of cancer, and me-as-him visiting her.\u00a0Unseen by me, except as her avatar in the goggles, a performer filled the role of the mother, occasionally taking my hand and responding to my responses with more or less scripted comments.<\/p>\n

I made my way to a cartoon bed and actually sat on it. The illustrated woman spoke to me, with Tannahill’s voice-over responding. On instructions, I went outside to fetch her paper, but coming back in I moved into a flashback of Tannahill (me) as a little boy playing with Mum. She chatted with me and I feebly responded, and followed her instructions to make colored marks on paper stretched out on the floor.<\/p>\n

\"Draw<\/a><\/p>\n

Later that night Tannahill’s narration guided me to a more unpleasant memory, one reminiscent\u00a0of Terence Davies’ Distant Voice, Still Lives<\/em>. It was a minimal narrative to match the images, but I could fill it thanks to\u00a0the familiarity of the scenario and the tactility of furnishings (bed, door) and physical human contact.<\/p>\n

The Deserted<\/em>, a 55-minute piece, was frankly billed as “a film by Tsai Ming-liang.” A series of fixed long-shots joined by cuts shows a typical Tsai situation: a young man and his mother live in a ruined temple. He apparently administers electrotherapy to his back and chest; he floats in his bath with a fish; it rains in buckets\u00a0and water comes seeping in to where you’re hovering. A woman in white, perhaps a ghost, appears at times, once in the\u00a0man’s bathtub.<\/p>\n

You make the choices in \u00a0La Camera Isabbiata<\/em>, and you role-play in Draw Me Close<\/em>, but in The Deserted<\/em> you’re locked down as a witness. No images from the piece seem to be available, but this, from a shot of a camera monitor, roughly\u00a0indicates\u00a0the film’s first shot (which isn’t as distorted as this looks).<\/p>\n

\"Deserted<\/a><\/p>\n

You can’t move to another spot, and you can’t even crane your neck to peek around corners. You can swivel your head\u00a0to look\u00a0360 degrees horizontally and vertically.\u00a0But nothing much happens behind your back or over your head. All the action, such as it is, takes place squarely in front of you. Given Tsai’s penchant for static long takes and deep areas of space, The Deserted<\/em> isn’t putting you inside a real or virtual space; it’s putting you inside a Tsai film.<\/p>\n

 <\/p>\n

Reality has the most bandwidth<\/strong><\/p>\n

Since I’m no expert on VR, all I have are initial observations\u00a0based on this sampling and on a bit of reading.<\/p>\n

First observation:\u00a0VR doesn’t have to be photorealistic to engage you. The image in Draw Me Close<\/em> is quite impoverished in terms of real-world cues, and the shots of The Deserted<\/em> are almost shockingly fuzzy. (Tsai, a fan of razor-sharp focus, would never let them in one of his film<\/em> films.) Other pieces in Venice VR, visible on monitors while users were getting the full dose, are often quite cartoonish. Why are they then so compelling?<\/p>\n

E. H. Gombrich suggested long ago\u00a0that illusion can rely more on stimulation than simulation. That is, choose just a few sensory triggers and you’ll be aroused even if the image isn’t particularly lifelike. Baby geese can, if primed properly, take a box for their mothers, and frogs can snap at anything floating in a fly-ish sort of way around their heads. Outlines, it seems, are central for creatures like us, but they don’t have to be detailed. We also respond, fast and stupidly, to\u00a0configurations that suggest faces.<\/p>\n

\"Tortilla\u00a0 \u00a0 \u00a0\"Butterfly<\/a><\/p>\n

Try not<\/em> to see Jesus, or Janis Joplin, in the tortilla, or a stare\u00a0on the wings of a Caligo butterfly.<\/p>\n

As caricaturists and camouflagers know, a few distinctive features can suffice to summon up the target, and the same goes for the partial and degraded input we see in Draw Me Close<\/em> and The Deserted<\/em>. In addition, concepts play a role; our ideas about what to expect in certain surroundings, such as a household, help the illusion cohere. And the information needs to be consistent. When I turn my head in Tsai’s temple, nothing I see contradicts the other cues.<\/p>\n

A second point: The VR experiences subtract on some channels of input and compensate on others. Normally, perceptual reality is massively redundant. Our sensory systems reinforce one another. Tilting your head sends signals about balance and vision, while sight and sound and touch and muscular activity offer mutually confirming information.\u00a0And when things are uncertain, you can always test your environment by moving around.<\/p>\n

But the VR systems I encountered impoverish the input on certain channels. Standing in the helicopter of Greenland Melting<\/em>, I saw the passenger seat beside me as vividly three-dimensional, but when I tried to grab it for support, nothing was there. In The Deserted<\/em>, I looked down and didn’t see my feet; in fact I was an eye in a bubble floating midway up the wall, where the camera was. And in neither of these pieces could I move voluntarily to poke around\u00a0much in\u00a0the space.<\/p>\n

I could do that in La Camera Insabbiata<\/em>, but it was an impoverished environment along other parameters. I could fly, but not touch any of the surfaces–which were without color\u00a0or shadow, beyond a spotlight that shifted to follow my attention\u00a0(presumably guided by eye-tracking software). I couldn’t land atop one one of the monoliths, or bump into an edge.<\/p>\n

Reciprocally, Draw Me Close<\/em> provided other sensory inputs: locomotion (though not wholly voluntary), touch (initiated by the performer playing Mum, and by the prop bed and door), and an array of skimpy visual-spatial cues. In principle, smell could have been included. Why not, then, go the whole hog and use a photorealistic rendering of the space and Tannahill’s mother?\u00a0Perhaps\u00a0a very faithful\u00a0representation of Mum and the household\u00a0would have raised the problem of the Uncanny Valley<\/a>.<\/p>\n

Just as important, we might ask: Since a performer is always required for the piece, why not simply play out\u00a0the scenes\u00a0with her and the user? Just go for reality, not virtual reality. I’m guessing that the move toward\u00a0drawn\u00a0imagery marks the project as stylized enough to be a media artifact, rather than a piece of interactive theatre. (Though it is partly that.) Here, I’m thinking, the impoverishment of information on certain channels was a deliberate aesthetic choice, rather than being obligated by computer processing power,\u00a0though that would probably have been a factor too.<\/p>\n

One of the biggest compensations for what’s missing in some channels would seem to be good old peripheral vision. This is a very strong cue to immersion, and it can override inputs that contradict it. I remember being in one of the Disney World attractions back in the 1980s, a wraparound theatre that put visitors at the center of a travelogue. Even though there were plenty of cues that you\u00a0weren’t in front of Buckingham Palace–such as your awareness of people around you looking in different directions–if you concentrated on the filmed display, there was a compelling illusion that you were riding\u00a0down the street toward the Palace.<\/p>\n

Peripheral vision would seem to be a powerful sensory trigger for creatures like us. This was the insight that drove widescreen cinema and the curved screen of Cinerama. But efforts to activate peripheral vision\u00a0maximally come\u00a0at a cost. You lose the frame and thus a sense of significantly composed imagery. The Disney display had, as I recall, borders at top and bottom, but in modern VR those too are abolished. VR gives with one hand and takes away with the other, so to speak: Greater immersion yields less of the pictorial structuring that’s inherent in framing. Tsai’s film somewhat overcomes this problem by making the surrounding space neutral and fairly uninformative, but the result still loses the sharp dynamic of offscreen and onscreen space we find in his feature films.<\/p>\n

One more thought, which seems more solid than the others. Some have asked whether VR can accommodate narrative. I had presumed\u00a0so theoretically, but now I’m convinced. Greenland Melting<\/em> told a real-world story, an alarming one at that. Draw Me Close<\/em> had emotion-laden scenes, a crisis and climax, and a flashback. The Deserted<\/em> presented a typical Tsai scenario of humid stasis–a thin\u00a0narrative, but a narrative. (As often in Tsai, what we miss in psychology and causal density we get in slight spatial changes and pictorial\u00a0surprises.) It\u00a0wouldn’t be\u00a0hard to “narrativize” the\u00a0Camera Insabbiata<\/em> soaring, either, say by prodding me to go on a cosmic scavenger hunt. I conclude that of course VR can deal with narrative.<\/p>\n

John Landis worried that VR was too wedded to long takes to suit cinematic storytelling. That worry was in turn tied to the assumption that narratives must guide your attention. True! But it’s wrong to assume that only continuity editing can shape a film story. Our attention can be precisely guided through long takes. Indeed, to a degree the sort of open and exploratory attitude some find in VR was already there in locked-down long-take directors like Mizoguchi, Akerman, Hou, and Tsai. And they told stories, plenty of good ones.<\/p>\n

\n

Many thanks to\u00a0Peter Cowie, Alberto Barbera, Michela Lazzarin, and all of their colleagues for inviting and assisting us. Special thanks to Andrea Vesentini for guiding us through the array of events at Venice VR.<\/p>\n

For details on Venice VR, go here<\/a>. Variety<\/em> covers the Venice event in articles by Elsa Keslassy<\/a> and Nick Vivarelli<\/a>. Patrick Frater interviews Tsai on The Deserted<\/em> here<\/a>. I found this Wired piece<\/a> a good VR \u00a0primer, and Ty Burr offers an update in\u00a0<\/em><\/a>MIT Technology Review<\/em>.<\/a><\/p>\n

Gombrich’s key essay on simulation and stimulation is “Illusion and Art,” available without illustrations here<\/a>. In full form it appeats in Illusion in Nature and Art<\/a><\/em>, edited by Gombrich and Richard L. Gregory (Scribners, 1973), 193-243.<\/p>\n

For discussions of how storytelling cinema guides attention within the shot, see the categories Film Technique: Staging<\/a> and Tableau Staging.<\/a><\/p>\n

\"VR<\/a><\/p>\n

Better than a cellphone? Viewers at Venice VR.<\/em><\/p>\n","protected":false},"excerpt":{"rendered":"

La Camera Insabbiata (2017) by Laurie Anderson and Hsin-chien Huang. DB here (still at the Mostra): Kristin and I are Virtual Reality novices, so when we got a chance to sample some pieces at\u00a0Venice VR, we jumped. The demand was great for the 22 pieces on display,\u00a0and you had to book each encounter separately in […]<\/p>\n","protected":false},"author":1,"featured_media":0,"comment_status":"closed","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[32,237,1,59,236],"tags":[],"class_list":["post-37625","post","type-post","status-publish","format-standard","hentry","category-directors-tsai-ming-liang","category-festivals-venice","category-film-comments","category-technique-staging","category-virtual-reality"],"_links":{"self":[{"href":"https:\/\/www.davidbordwell.net\/blog\/wp-json\/wp\/v2\/posts\/37625","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/www.davidbordwell.net\/blog\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/www.davidbordwell.net\/blog\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/www.davidbordwell.net\/blog\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/www.davidbordwell.net\/blog\/wp-json\/wp\/v2\/comments?post=37625"}],"version-history":[{"count":28,"href":"https:\/\/www.davidbordwell.net\/blog\/wp-json\/wp\/v2\/posts\/37625\/revisions"}],"predecessor-version":[{"id":37688,"href":"https:\/\/www.davidbordwell.net\/blog\/wp-json\/wp\/v2\/posts\/37625\/revisions\/37688"}],"wp:attachment":[{"href":"https:\/\/www.davidbordwell.net\/blog\/wp-json\/wp\/v2\/media?parent=37625"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/www.davidbordwell.net\/blog\/wp-json\/wp\/v2\/categories?post=37625"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.davidbordwell.net\/blog\/wp-json\/wp\/v2\/tags?post=37625"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}