The 43 muscles in your face can form 10,000 unique expressions. Ehsan Hoque, a 2004 graduate of Penn State Erie, The Behrend College, programmed a computer to interpret all of them – and then teach you to do better.
The program can be particularly useful for people with Asperger’s syndrome, who often have difficulty reading emotions and other social cues. It also can benefit sales and customer service reps, students prepping for class presentations or job applicants rehearsing for an interview, according to MIT Technology Review, which recently chose Hoque as one of the top 35 innovators under the age of 35.
“A computer is completely objective, standardized and non-judgmental,” said Hoque, an assistant professor of computer science at the University of Rochester. “It’s a safe place for people with social anxiety or awkwardness to practice and learn.”
MIT Technology selected Hoque for his work with two computer technologies: MACH (My Automated Conversation coach) and RocSpeak.
For MACH, he created a virtual businesswoman programmed to recognize the user’s expressions and statements. She nods, smiles and asks questions, all the while noting the user’s body language, intonation and eye contact.
More than 20,000 people have used the program to improve their social skills since Hoque debuted it in 2015. More than half have agreed to share their results for research purposes, allowing Hogue to further refine the program.
“Building the original platform was easy, but then you have to add in all the insights,” he said. “That never really ends. It is an ongoing, evolving process.”
RocSpeak is a pared-down, mobile version of the MACH program and is available at no cost online. There is no animated assistant; instead, the program records video and sends users a report on various aspects of their interpersonal performance, such as speed of speech, pitch and smile intensity.
Hoque experimented with speech and sight recognition while at Penn State Behrend. His senior design project team created a robot that could see, hear and recognize people. It won the “Best Design” award.
Years later, as a doctoral student at MIT, he presented his work on a computer program that could gauge a user’s mood. After the demonstration, an audience member approached him, asking if he might expand the program.
“He said he was very awkward in social situations, and he wondered if my technology could be used to create some sort of automated program that he could use at home to practice making eye contact and improve his social skills,” Hoque said.
Hoque understood the challenge. He is the primary caretaker for his brother, who has Down syndrome, a genetic condition that affects approximately 400,000 Americans. That provided additional motivation to create the MACH program, and then RocSpeak.
“I have these advanced degrees and technical skills,” Hoque said. “If I can use those to solve problems for people, then it is knowledge and time well spent.”
