The Autism Glass Project Could Help Kids Better Recognize Facial Expressions

 

Autism is, in part, about “delays in communication and socialization,” according to Judah B. Axe, an associate professor of behavior analysis at Simmons College. In 2012 Axe and Christine J. Evans studied how children with autism respond to eight facial expressions (approval, bored, calming, disapproval, disgusted, impatient, pain, and pleased), primarily through the use of video. Previous research had focused on using static pictures or puppets.

As the autism community celebrates the 47th annual Autism Awareness Month, a developing technology called Autism Glass is hoping its facial recognition software can build on Axe’s and other researchers’ work on facial expression therapies for children with autism.

“Responding to people’s facial expressions is necessary for observational learning, showing empathy, and other social processes, and children with autism exhibit delays and deficits in responding to people’s faces and emotions,” Axe and Evans write in “Using video modeling to teach children with PDD-NOS to respond to facial expressions” in the journal Research in Autism Spectrum Disorders.[1]

Autism Awareness Month

There has been a mountain of research on autism spectrum disorder (ASD) since the Autism Society declared the first Autism Awareness Month in 1970. Today, about 1 percent of the world population has been diagnosed with ASD, according to the Centers for Disease Control and Prevention. In the U.S., 1 of every 68 school-aged children has ASD, up nearly 120 percent since 2000 (when it was 1 in 150 children).

Scott Badesch, CEO of the Autism Society, explains Autism Awareness Month plays a critical role in the quality of life for those living with autism.

“When we say ‘quality of life,’ we’re talking about basic human rights that allow people to interact with one another and the world on their own terms,” Badesch says. “Autism Awareness Month is an opportunity to share our vision, where individuals living with autism are able to maximize their quality of life and live in a society in which their talents and skills are appreciated and valued.”[2]

Autism Awareness Month also shines a light on new developments in autism intervention. Because of their use of burgeoning technologies, researchers at The Wall Lab at the Stanford School of Medicine have an especially bright spotlight shining on their work. Dennis Wall, who leads the project, and his team are close to being able to unveil Autism Glass to the public.

Using Google’s Technology

Autism studies have shown that children with autism have a difficult time recognizing emotions in other people or responding to even common facial expressions like happy, sad, or angry. The Wall Lab built its emotion-recognition software to work with Google Glass to translate facial expressions, in real time, to an autistic child wearing the device. 

Researchers have found that when looking at faces, “typically developing individuals commonly look at people’s eyes whereas individuals with autism look at people’s mouths and inanimate objects,” Axe and Evans wrote in their 2012 paper. “To complicate matters further, facial expressions often have durations lasting microseconds, and children with autism have more difficulty recognizing faces when presented rapidly.”

Autism Glass is designed to leverage technology to diminish the reliance on less scalable aspects of applied behavioral analysis (ABA) methods, for example including the use of flashcards to teach emotions, which often don’t generalize well to the child’s natural environment. When someone the child is interacting with reacts with a facial expression conveying emotion, an indicator in the form of an emoticon, a color or a word appears on a small display above the right eye.

The glasses are paired with software, via a smartphone app, that analyzes data coming in from the child’s device and records what the child is seeing and measures her ability to read facial expressions. The goal is to make sure the glasses are a learning aid or coach, not a prosthesis.[3] The hope is that children can become more socially confident and feel less isolated.[4] 

Ericka Wodka, a pediatric neuropsychologist at the Center for Autism and Related Disorders, explains how autism intervention techniques feature both low-tech strategies (a binder filled with pictures for the child to point to) and high-tech strategies (iPad apps).

Every child is different of course, so the type of technology that works with each differs. And yet technology itself “is highly motivating for these kids,” Wodka says. “They’re motivated to use technology, so that helps meets the end goal. The flip side, obviously, is that technology can be distracting if it’s not set up in a way that’s appropriate for a child.”

Reducing Diagnosis Time

Wall and his fellow researchers hope their Autism Glass will be that appropriate technology. In 2012, when Wall was director of the Computational Biology Initiative at the Center for Biomedical Informatics at Harvard Medical School, he designed an algorithm that reduced the time for an autism diagnosis by nearly 95 percent, from hours to minutes.[5] Today, Wall says he sees his team’s work with Autism Glass as a necessary follow-up project to his work in diagnostics.

“Once a diagnosis happens, there’s a bottleneck to therapy — too many kids and too few clinicians,” Wall says. “I don’t want to succeed with mobile diagnostics without also mobilizing therapy ... . I worry about doing well with one side, but not the other. Our glass solution is one of a panoply of mobilized mechanisms to get families and behavior technicians a faster and better way that doesn’t require intensive clinical visits." 

Wall says his lab is working with behavior technicians as it refines Autism Glass, and hopes to give behavior professionals “another tool, that’s exciting for the child and lets behavior technicians deliver the emotion-recognition components of ABA.”

Capture the Smile

In one game that’s part of the glass software, called “Capture the Smile,” the child is tasked with getting the behavior technician to make a happy face by making a joke or a funny face.

“It’s a second layer of learning,” Wall explains. “First they associate a word to a face the behavior technician is making, and next, the feeling behind that emotion. How do we engage the child’s brain development to capture the physiological feeling that underlines the name of that emotion?” He adds that parents can also participate, and give continuity of care between every visit with the behavior technician.

Wodka says kids with autism learn best when they have specific exposure, and specific feedback. Autism Glass, she explains, promises “an opportunity to provide repeated feedback in a social situation.”

“The other promise is that the Glass will help them decrease ambiguity of social interaction in real life,” Wodka says. Kids struggle a lot with skills they practice in therapy sessions when they try to apply them in real life. With the Glass, the practice is happening in real life — it can be a bridge.”

Wall says the Autism Glass project is in the middle of a trial designed to measure how much the device has helped once a child is no longer using it, a move toward allowing the glasses to be reimbursable by insurance companies. He shares that he hopes “we move it out of the lab before end of the year,” and onto the market in the first half of 2018.

Simmons professor Judah Axe said a delay in recognizing others’ emotions is “a defining characteristic of autism spectrum disorder, and often very difficult to remediate.”

Autism Glass, he continued, “is a promising device that can be incorporated into ABA programs to prompt responses to facial expressions and improve the social functioning of individuals with autism.”


[1] Axe, Judah B. and Christine J. Evans. “Using video modeling to teach children with PDD-NOS to respond to facial expressions.” Research in Autism Spectrum Disorders, Page 1176. 2012.

[2] Badesch, Scott. Emailed statement to BehaviorAnalysis@Simmons. March 10, 2017.

[3] Patel, Prachi. “Autism Glass Takes Top Student Health Tech Prize.” Scientific American. April 12, 2016. Retrieved March 25, 2017 from www.scientificamerican.com/article/autism-glass-takes-top-student-health-tech-prize-slide-show1

[4] Hoshaw, Lindsey. “Google Glass Flopped. But Kids With Autism Are Using It to Recognize Emotions.” KQED Science. June 23, 2016. Retrieved March 25, 2017 from ww2.kqed.org/futureofyou/2016/04/08/google-glass-flopped-but-kids-with-autism-are-using-it-to-learn-emotions

[5] DuBoff, Katie. “Detecting autism in matter of minutes.” Harvard Gazette. April 10, 2012. Retrieved March 25, 2017 from news.harvard.edu/gazette/story/2012/04/detecting-autism-in-matter-of-minutes