Sadness Detection for Future Smart Homes

Abstract

This thesis focuses on sadness detection and recognition using deep learning and image processing in python. It analyzes accurate and efficient ways to collect a large set of “moments” from YouTube videos to build large-scale databases for “moments” that show the emotion of sadness. For the overall model architecture, a sequential neural network model is built with three fully connected convolutional layers and rectified linear units as our activation function. Initially, we obtain a nearly zero false positive rate and around ten percent false negative rate on this trained model. To further improve the accuracy and efficiency, the Haar Cascade classifier is used to crop only frontal face images and OpenPose is used to locate facial key points to precisely detect and analyze the facial expression. Besides, we crawl the YouTube network to acquire the video information and used natural language processing to filter the videos that are more likely to contain the emotion sadness. By incorporating the deep learning model with the above algorithms, “moments” that contain the emotion of sadness are extracted from YouTube videos and output as a JSON file, which can be viewed via the iLab AI-Human Video Database.