Research, Critique & Reflection

RESEARCH

Below I will address some further questions relating to this Master’s Project from the University’s internal Learn page.

What are your influences & inspirations? (Specifically, list a few projects, papers, theories)

Content Influences:
Irish myths and Legends
“The Tain” (primarily both ThomasKinsella’s 1969 version and Ciaran Carson’s 2007 version.

Broader Fairy tales, myth, and folklore from around the world as a means to better understand indigenous cultures.

Project / Concept Influences:

‘Chalkroom’ (2017) by Laurie Anderson and Hsin-Chien Huang

“Easter Rising: Voice of a Rebel” VR (2017) from BBC Taster.

“The Book of Distance” (2020) by Randall Okita

A Jester’s Tale

Quill 2.0: The VR Storytelling Tool

Academic influences:

In regards to the last reference above, this gave me an idea on how to link these very entangled stories via the use of portals in VR. Whilst not directly the point of the paper by Pádraig Mac Carron and Ralph Kenna’s, looking at the relations of these stories using the analogy of a neural network is actually very useful and provided an idea on how one may navigate through related stories, provide greater context and keep everything diegetic and without breaking immersion.

Other Aesthetic Influences:

Whilst my direction may change here as things come together more, two main early influences have been:

- Gnomes & Goblins VR (in development) by Jon Favreau in collaboration with Jake Rowell.

Excalibur (1981) John Boorman

Whilst to an extent these represent a more naïve realism I believe, as Abigail Salvesen also concluded (The Red Room – Exploring narrative experience in the adaptation of fictional literature to virtual reality) this style enables greater immersion without detracting from the story. Nonetheless I want to play around with shaders and lighting as much as possible to create a more unique identity.

What are the main issues and debates in this subject area?

The Narrator as a Diegetic element: Typically, the narrator in a spoken word story is a non-diegetic element. In this case, the storyteller is a diegetic element and a focal point. The challenge will be to balance this with other interactions and diegetic affordances, allowing the user to navigate the experience without distracting from the story or confusing.

VR is “Not for storytelling”: Much has been written in the tech media concerning how VR is “not for traditional narratives”. I believe this was a perspective from a time (even though a few years ago) when the medium had not really been explored in any great depth

TechCrunch, 2016: “VR is terrible for traditional storytelling”

The Guardian, 2015: “Pixar co-founder warns virtual-reality moviemakers: ‘It’s not storytelling‘

TechInAsia, 2016: “VR will not be the new paradigm for storytelling. Here’s why”

Whilst I would agree that you need to adapt to the medium, I think it is very possible to tell a traditional story in VR. Which brings me to another challenge:

Building a visual environment around audio that is not “made for purpose”: I will be experimenting with AI and voice cloning for certain actors and guides (another diegetic affordance) within the environment, some of the audio I will be dealing with has been recorded many years ago and was not intended to be presented as a VR story. Building this into the worlds I’m creating without making a confusing mess will be challenging.

Which are the three most important papers that relate to your topic?

Senehi, J (2010). Storytelling and Peace
Shin, D (2018). Empathy and embodied experience in virtual environment: To what extent can virtual reality stimulate empathy and embodied experience?
Bucher, J. (2018). Storytellingfor Virtual Reality. Methods and Principles for Creating Immersive Narratives.

What is the relevance of your work to other researchers?

The primary relevance, I hope, will be in exploring how one can take a traditional narrative where the storyteller is a diegetic element, and present this in VR in an engaging manner. Particularly when dealing with audio stories, not intended for the VR medium.
Those exploring how to further the voices and understanding of indigenous communities and different cultures by sharing stories on a global platform.
Those exploring Voice Cloning / SSML and the use cases for same, particularly in relation to storytelling.

What is the relevance (if any) of your work to practitioners?

Similar to the above as we are dealing with a still relatively new medium, but more focusing on the practical and less on the aesthetic. Meaning is SSML a viable solution for commercial VR / gaming projects? How can one reuse old audio recordings as part of a VR narrative?

Critique & Reflection

Summarise the key findings from your literature research so far.

Multi-modal / multi-sensory is the way to go when building immersion in VR.
Whilst AI Voice cloning solutions are impressive, voice Cloning solutions must further evolve before they can be useful in a commercial context where emotional intonation matters (e.g. outside of a support chat bot).
In relation to the above, SSML is limited and needs to evolve.
“Out of the box“ AI /Voice synthesis models are poor when dealing with regional accents
Reuse of sets follows a theatre principal to a large extent
Non-diegetic elements kill engagement with the story.
The use of metaphor in affordances is important when creating diegetic interfaces and ensuing immersion is maintained.
Many sources have cstated that “VR is not for storytelling” or “Not for traditional narratives”.

Which of these findings are the most interesting to you? Why?

Most intriguing to me is how to create diegetic affordances in a VR world of mythology that still remain intuitive to a user. 😊

Also of interest is the possibilities (beyond the scope of this project) of natural language processing / AI that could interpret stories, poems, descriptions of places, people, things and build that in a procedural sense. Almost like the Shakespearean sonnet writing AI reversed. The idea of describing a dream and having that world brought to life around you.

Which do you most disagree with? Why?

I strongly disagree with the numerous tech media articles (more so from the earlier years of this iteration of VR), that VR is not for storytelling. This perspective is not as prevalent in academic works. I would imagine this is probably because it was necessary for the medium to evolve and for more research / exploration before coming to such a conclusion.

I would say the evolution of AI driven voice-cloning solutions is overstated. Perhaps not from an academic perspective, but certainly from an industry perspective.

Which are the best three examples you have found (that you didn't know before) of work related to what you want to do?

Mentioned above:

“Chalkroom” (2017) by Laurie Anderson and Hsin-Chien Huang.
“Easter Rising: Voice of a Rebel” VR (2017) from BBC Taster.
“The Book of Distance” (2020) by Randall Okita

Which are the worst three?

I cannot honestly designate any 3 “worst” examples specific to VR storytelling (not gaming). Even in the above 3 “best examples” there are elements I admire greatly and things I would do differently.

Mostly in terms of moving the user from observer passive to observer active through greater levels of interaction. These need not be ugly non-diegetic interfaces, but VR is a medium of interaction and I think some opportunities were missed in the above examples. I respect that there is an argument for compelling the viewers attention, but that should be a natural result of good design, not forced lack of interaction.

Having said that…. I did come across “Myth: A Frozen Tale” from Disney (2020) on Occulus. I was left baffled. The animations were quite beautiful, but the whole experience seemed rough and unfinished, lacking in magic. Some events seemed random as was the positioning of the viewer / camera. And the lack of interaction made me think my controllers were not functioning.