Exploring Effective Storytelling Guidelines for Cinematic Virtual Reality

Summary

This PhD research dives into the complex realm of cinematic virtual reality (CVR) to address challenges and offer innovative solutions for creating immersive storytelling experiences. The emergence of CVR as a potent storytelling tool raised initial concerns related to the narrative paradox (NP) and the fear of missing out (FOMO). Content creators initially grappled with adapting storytelling techniques from traditional media like cinema and theater into the unique VR landscape.

To explore these challenges and potential resolutions, a series of user studies were conducted as part of this thesis. One significant development was the introduction of Action Units (AUs), a human body-language-based attention guidance cue set. AUs were designed to counteract the FOMO issue by enhancing viewer engagement and enjoyment in CVR content. The study compared AUs with other commonly used synthetic cues, with findings showcasing AUs as favorable for their diegetic qualities and ease of use, both for creators and viewers.

The second user study focused on understanding the viewer’s desire for active interaction in narration. The results emphasized the importance of viewer control in CVR projects, indicating that allowing viewers agency enhances their engagement. To guide content creators in navigating these challenges and incorporating the findings, the research led to the development of the Adaptive Playback Control (APC) framework.

The APC framework provides a comprehensive guide for CVR creators, addressing FOMO and enhancing viewer interaction through integrated personalization. It emphasizes diegetic attention guidance cues, encourages harmony between viewers and creators in directing narrative development, and underscores the visibility of interaction affordances in immersive storytelling experiences.

A real-world case study applied the APC framework to an immersive Māori storytelling experience, demonstrating its adaptability and cultural appropriateness. Personalized variations were introduced, considering the unique demands and participation levels of individual viewers. This case study highlighted the importance of early consideration for individual users in the content design process.

In conclusion, the APC framework stands as a novel and invaluable tool for CVR creators, offering guidance from content preparation to interactive design. It not only resolves the challenges of FOMO and NP but also elevates the overall viewer experience. The framework’s adaptability, showcased in a real-world case study, positions it as a valuable asset for creators across diverse contexts, extending beyond the cultural heritage sector.

Acknowledgement

The researchers in this project would like to thank the support of researchers from the HIT Lab NZ, University of Canterbury, Prof. Hēmi Whaanga, Dean Whaanga, the late Bubba Thompson, and others from Te Rūnanga o Awarua, Te Rau Aroha Marae in Bluff, and research members from the New Zealand Science for Technological Innovation National Science Challenge (SfTI NSC) project Ātea.

E te hoa, te rangatira Bubba, te uri o Ngāi Tahu me Ngāti Kahungunu. I haoa koe ki te kupenga nui a Taramainuku, i whakairia ake ki te hao o Rua, ā kua whetūrangihia koe. E tangi tonu ana te ngākau, e auē tonu nei te mamae. Nō reira, moe mai rā, oki oki oki atu.

Researcher and Contact

Kris Tong

Supervisor Team

• • Rob Lindeman (senior supervisor), HIT Lab NZ, University of Canterbury
  • Holger Regenbrecht (co-supervisor), Department of Information Science, University of Otago
  • Rory Clifford (co-supervisor), HIT Lab NZ, University of Canterbury
  • Richard Chen Li (previous co-supervisor), Department of Applied Social Sciences & Department of Computing, The Hong Kong Polytechnic University
  • Sungchul Jung (previous co-supervisor), Kennesaw State University

Research Outputs

Thesis

• • Tong, Lingwei (2022). Exploring effective storytelling guidelines for cinematic virtual reality [Doctoral thesis, University of Canterbury]. UC Research Repository.http://dx.doi.org/10.26021/14309

Journal articles

• • Tong L, Lindeman RW, Lukosch H, Clifford R, and Regenbrecht H. (2024). Applying Cinematic Virtual Reality with Adaptability to Indigenous Storytelling. ACM Journal on Computing and Cultural Heritage. Just Accepted (February 2024). https://doi.org/10.1145/3647996
  • Regenbrecht H, Park N, Duncan S, Mills S, Lutz R, Lloyd-Jones L, Ott C, Thompson B, Whaanga D, Lindeman RW, Tong K. (2022). Ātea Presence— Enabling Virtual Storytelling, Presence, and Tele-Co-Presence in an Indigenous Setting, IEEE Technology and Society Magazine,  41 (1), pp. 32-42, March 2022, doi: 10.1109/MTS.2022.3147525.
  • Tong L, Lindeman RW, and Regenbrecht H. (2022). Adaptive Playback Control: A Framework for Cinematic VR Creators to Embrace Viewer Interaction. Front. Virtual Real.2:798306. doi: 10.3389/frvir.2021.798306
  • Tong L, Lindeman RW, and Regenbrecht H. (2021). Viewer’s Role and Viewer Interaction in Cinematic Virtual Reality, Computers, 10(5):66, doi: 10.3390/computers10050066.

Conference proceedings

• • Tong L, Jung S, Li RC, Lindeman RW, and Regenbrecht H. (2020). Action Units: Exploring the Use of Directorial Cues for Effective Storytelling with Swivel-chair Virtual Reality. In 32nd Australian Conference on Human-Computer Interaction (OzCHI ’20). Association for Computing Machinery, New York, NY, USA, 45–54. https://doi.org/10.1145/3441000.3441063
  • Tong L, Jung S, and Lindeman RW. (2020). Action Units: Directorial Cues for Immersive Storytelling in Swivel-chair Virtual Reality. In 2020 IEEE Conference on Virtual Reality and 3D User Interfaces Abstracts and Workshops (VRW), Atlanta, GA, USA, 2020, pp. 277-278, doi: 10.1109/VRW50115.2020.00057.
  • Tong L, Jung S, and Lindeman, RW. (2019). Action Units: Directing User Attention in 360-degree Video based VR. In Proceedings of the 25th ACM Symposium on Virtual Reality Software and Technology (VRST ’19). Association for Computing Machinery, New York, NY, USA, Article 52, 1–2. https://doi.org/10.1145/3359996.3364706.

Project team at Te Rūnanga o Awarua, Te Rau Aroha Marae in Bluff

People