How Mixed Reality Can Help with Autism
You may have heard about mixed reality beginning to play a role in work industries and scientific research. With the introduction of Microsoft Mesh at the Ignite 2021 conference, we saw examples of holographic 3D structures and deep sea explorations taken on land. Those working in machinery can use virtual tools to augment their workflow in physical space. But there’s another application that many of us haven’t considered: how mixed reality might be able to help individuals with social or developmental challenges like autism. Actimage, a French innovation company, has set out to work on this specifically in its Hol’Autisme project.
About Actimage
Founded a little over 20 years ago in France, Actimage has branched out to other countries as well, including Germany, Switzerland and the United States. They specialize in technology innovation and strategy, and one of their areas of expertise is the HoloLens. They began creating applications for the HoloLens in 2018 and are now putting it to work in a variety of use cases. Their work with mixed reality coincided with a growing interest in health policies, and particularly autism, in the French government. So Actimage created its Hol’Austisme suite to help empower autistic individuals with life skills and practice opportunities to help them succeed in the real world.
HoloLens Features
Microsoft’s mixed reality headset is the lens through which users take part in what Actimage’s designers and developers have created. It’s an autonomous device that does not need to be attached to any computer; it can be connected to a Wi-Fi network or through Bluetooth. It makes use of sensors and infrared cameras to scan and pinpoint its location in space, allowing for interactive virtual elements added to the physical environment. It also has gesture and vocal recognition, which further enhance its capabilities to create a mixed world through the user’s normal movement and speech. A built-in 3D sound system tops off the immersive experience. When multiple users connect wirelessly, they can share a space made up of both the physical and the virtual.
Apps for Autism
Image from Actimage’s Smile video.
Actimage’s objective regarding autism is “to create a new approach to help children to identify and imitate the social codes that they aren’t mastering, to be mobile in noisy locations that make them anxious and inhibit them, and to help them acquire autonomy.” The Hol’Autisme suite is comprised of a handful of applications. One of the everyday social skills that autistic individuals can struggle with is identifying and responding properly to different facial expressions and emotions. The Smile app helps users practice viewing and correctly naming expressions, both in the form of holographic samples and tags added to real people the user interacts with. Through this kind of practice, autistic individuals can learn to make those connections logically until they begin to reach a more instinctive understanding like that of people who aren’t on the spectrum.
A more rudimentary app called Pop Balloons can be helpful for younger users or those with more severe autism: it gives them practice simply looking forward in front of them when they may be more inclined to look at the ground, with the objective of popping virtual balloons floating ahead of them. Another app named Across simulates a busy street intersection to get users in the habit of managing multiple stimuli safely.
Helping People Join Society More Fully
Image from Actimage’s Job Now video.
These exercises are largely aimed toward developing children, but the suite can be helpful for adults too. Many on the autism spectrum have trouble with the social experience of interviewing for a job, and failed attempts often lead to hopelessness and giving up. With the Job Now app, the process can be made much less daunting. Users practice in a simulated interview and then get feedback afterwards from a real person who reviews it. This human coach provides a summary of how the interview went based on a number of performance indicators like length of answers, interactive gestures, and conversational speed. Then the user can practice again until he or she feels more ready to take on actual interviews.
Broader Potential for Mixed Reality
While autistic individuals gain skills and confidence through these apps, anonymous usage data is also gathered for research on autism. The goal is thus to both help those on the spectrum and also gain a better understanding of the challenges they face, a feedback loop that can continue to improve the lives of those affected. And it’s easy to envision how these apps could help others without autism, too: many with other conditions could benefit from these kinds of exercises as well. The potential leaps to be made are enough to make recognizable smiles around the world.