Title Details
Role Lead Developer
Tools
Platform: Magic Leap 1 (spatial audio, digital objects)

Dynamic dialogue: Google Dialogflow v2

Pulse Sensor: Heart rate monitor

Length 25 hours
Features AI companion, Text-to-Speech, Dialogflow, Sound Design, & 3D Assets
Project Sixth Sense is an AR app to simulate schizophrenia symptoms in an authentic and respectful way to foster understanding and empathy.
Collaborators
Destiny Guzmán, Julia Scott, Tingru Lian, Lucas Wozniak

Demo

Motivation

Building empathy and awareness around psychosis is crucial to effective allyship and safe interaction with affected individuals. This project aims to immerse users in the experiences of someone with schizophrenia, bringing greater understanding to a wider audience.

Exhibitions

The App

To experience the app, download The Lab and use Device Bridge to load the application on your Magic Leap One device.

User Journey

In the app, the user embarks on a journey in the shoes of someone living with schizophrenia, guided by a supportive friend. Routine encounters of daily life spark auditory hallucinations, and the player must rely on their friend’s guidance to navigate these unsettling experiences.

Key Features

Development Process

I led the app’s development using Magic Leap and Unity, which included troubleshooting and optimizing our workflow for AR on Magic Leap. This involved:

Storyboard + Content Creation

Event 1: Alarm

Event 2: Book

Event 3: Video on TV

Challenges & Solutions

  1. Mabu and manifest.xml Issues: Working with Mabu and manifest.xml proved challenging, as the documentation on the Magic Leap site was sometimes confusing. I found more effective support and templates on GitHub to address these issues.
  2. Device Registration Constraints: Although we had three Magic Leap devices, registration limitations restricted us to one device during development, which required careful planning and collaboration.
  3. Time Constraints: We agreed on the final storyboard just 15 hours before the end of the jam, necessitating quick implementation and prioritization to meet our goals.
  4. First-Time Use of Magic Leap: I familiarized myself with the Magic Leap device within 24 hours, quickly adapting to ensure efficient development.

Read more

Presentation

Github Special Thanks to Destiny Guzmán Julia Scott Tingru Lian Lucas Wozniak