Health and Behavior: Detecting Emotion in Multi-Party Conversations

Mentor: Jason Hallstrom, Ph.D.
Scholar: Jacob Belga
Home Institution:University of Central Florida

Mentors: Jason Hallstrom, Ph.D., Paul Peluso, Ph.D.
Scholar: Hannah Gibilisco
Home Institution: Stetson University

Mentor: Jason Hallstrom, Ph.D.
Scholar: Serena Walker Jean
Home Institution: Lehigh University

With our primary goal of developing a therapeutic tool to computationally interpret emotions, the purpose of our research is to identify and extract key emotion features. The analyses we are focusing on are facial analysis, audio signal analysis and sentiment analysis. Our approach takes these different forms of analyses and utilizes each unique data point to accurately compute an individual’s emotion. From the extracted data, our tool is designed to determine one out of six emotions: anger, disgust, fear, happiness, sadness and surprise. The challenge we are overcoming, through our research, is performing real-time calculations and interpreting an emotion. With this problem in mind, we are implementing threading and a machine learning classifier to increase efficiency and accuracy with the available processing power. Upon completion of testing the classifier, we will have a user-friendly GUI that displays aspects of an individual’s face, the audio signal of their speech and a transcript of their sentiments with an assigned positivity rating.

Click here for the PDF presentation.   Click here to watch the video presentation.