What is this?

Hi there! This is a rough journal of my research and production notes while working on my Master of Design in Animation at the Glasgow School of Art. For my final project I have to conduct independent research and present a written piece on a topic of my choice and also create a related piece of animation.

This is the original research proposal from last month. It has morphed a considerable bit since then, but I think it's good to know where it started:

= = = = = = = = = = = = = = = = = = = = = = =

17 May 2013

Research Proposal
Mediating Reality: Seeing the Unseen Augmentations

When faced with the term “augmented reality” many might envision holographic displays hovering mid-air as a science fiction hero interacts with it directly using hand motions. Others might point out the yellow line indicating a “first down” on an American football broadcast or flags following Olympic swimmers in their lanes indicating their current standing and time elapsed. It is commonly understood as experiencing a physical, real-world environment possessing elements that are supplemented by computer-generated input. It does not necessarily seek to duplicate the world’s environment in a computer or create an entirely separate iteration as does virtual reality. We live in a digital age where interacting with hardware and seeing results appear on a screen are so commonplace that it is nigh impossible to avoid in day-to-day life.

Because of the way we now inherently accept the digital, where does this place the boundaries for what is augmented from our daily experienced world? Are we even aware of this acceptance? Car commercials feature what seem to be actual vehicles moving through country roads or cities at night, but really they are more often than not computer-generated imagery meant to enhance our perception of the actual car. Have we been deceived, or have we merely accepted that what we are shown must be real and created for ourselves a “real” perceived world? The same concept can be applied to cinema. When we watch a movie, we are willfully suspending disbelief and entering the world that is shown to us on the screen. Advances in the technology of filmmaking have greatly increased the scope of worlds that we are able to enter. What started with manually compositing images with mattes then became green screen and chroma-keying. Really, though, is there so much different between Who Framed Roger Rabbit? and Avatar insofar that each takes our interaction and augments it not once through the mere watching of a motion picture, but twice through creating another layer, a world within that cinematic world, that we are asked to believe is whole and experience it as such? So much today is processed with digital cameras that the claim to create a commercial film with “no CGI” is laughable. I love the tangibility and character of traditional film as much as the next cineaste, but today's industrialized approach to filmmaking heavily tends toward the computerized, though I will not call it automated. There is definitely still an art to digital fabrication, and visual effects artists create explosions, the aforementioned cars, and fantastical creatures that we are asked to accept as physical and real, and, more often than not, we do. When a character exists as a physical puppet, digital double, voice actor, and countless animators, though, in what space does the character truly exist and become, as we experience her, real? Most effects are invisible to us since they are designed to be so; they are not meant to distract from the world we are shown, but to augment it.

I posit that the scope of what we describe as augmented reality is, in actuality, far broader than the common description. In a way, it is much more basic, and strangely enough the increased technological literacy of our world has led me to realize that even rudimentary tools have augmented our reality a very long time. I want to examine the blurry intersection between what we will accept as reality and those experiences that we hold at arms length and explain away as bits of code flying through an imagined space. At what point does a scene created in Maya stop being a simulation of 3D space and start being an actual environment that, due to our physical fleshy limitations, we must interact with through a computer? Haptics let us interact with digitally created objects, but at what point does the suggestion of an object become a “real”?

This research will be largely textual. I will create an accompanying animated short, but it will complement the written research rather than be a direct application of the research’s outcome, as that would require technology and skills to which I just do not have access. (Hopefully, though, my research could lead to eventual later application of principles learned.) The animated piece will allude to the boundary between what is perceived and experienced and what is real, though that is really all dependent upon the viewer. I would like to combine numerous media for animation, like projecting 2D image sequences onto 3D surfaces in Maya, as it adds further dimensions of augmentation and composite pieces of reality that we as the audience must synthesize in our brains into the experienced world.

As far as text goes, the main question I want to answer is because of the way we now accept the digital, is all digital cinematic interaction, then, augmented reality? Or has the digital merely become an extension of ourselves in a sort of transhumanist ‘cyborg’ kind of way? In a twist of etymological irony, we call something “digital” because it is numerically based coded information, and numbers are “digits” because we count them on our own digits, our fingers. So really, we’re just coming full circle and re-accepting our artifice back into ourselves. Additionally, what is a brain but a switch that turns muscles on or off in rapid succession to produce movement, electrical impulses, and other chemical responses? So, then, we are now re-learning how to interface with computers, our out-of-body brains, which turn on or off binary switches telling a computer what to execute next. I will supplement my more artistic take of the cinematic world of augmented reality by looking at the work of Steve Mann, applications of Natural User Interface, and work published at the MIT Media Lab in Fluid Interfaces and the Center for Future Storytelling. With luck, I will also be permitted to shadow the research being conducted upstairs at the DDS.