Facial Expression Recognition Using Deep Neural Networks

A Project Report On Facial Expression Recognition Using Deep Neural Networks


Automated Facial Expression Recognition (FER) has remained a difficult and interesting drawback. Despite efforts made in growing varied strategies for FER, exist- ing approaches historically lack generalizability when applied to unseen photographs or those which are captured in wild setting. Most of the present approaches are based mostly on engineered options (e.g. HOG, LBPH, and Gabor) the place the classifier’s hyperparameters are tuned to offer finest recognition accuracies across a single database, or a small assortment of comparable databases.

Don’t waste time Get a verified expert to help you with Essay

Nevertheless, the outcomes usually are not significant when they are applied to novel data. This paper proposes a deep neural network architecture to address the FER downside across a quantity of well-known standard face datasets. Specifically, our network consists of two convolutional layers each followed by max pooling after which 4 Inception layers. The network is a single component architecture that takes registered facial pictures because the enter and classifies them into both of the six fundamental or the impartial expressions.

We carried out complete experiments on seven publically obtainable facial expression databases, viz. MultiPIE, MMI, CK+, DISFA, FERA, SFEW, and FER2013. The outcomes of proposed structure are similar to or higher than the state-ofthe-art methods and better than conventional convolutional neural networks and in both accuracy and coaching time. This paper presents a novel deep neural community architecture for the FER downside, and examines the network’s capacity to perform cross-database classification while train- ing on databases which have limited scope, and are often specialised for a number of expressions (e.

g. MultiPIE and FERA). We carried out comprehensive experiments on seven well- known facial expression databases (viz. MultiPIE, MMI, CK+, DISFA, FERA, SFEW, and FER 2013) and procure results that are considerably higher than, or comparable to, traditional convolutional neural networks or different state-of- the-art strategies in each accuracy and learning time.


We have immense pleasure in profitable completion of this project on Facial Expression Recognition Using Deep Neural Networks. We want to take this opportunity to express our gratitude to Dr. C P S Prakash, Principal of DSCE, for permitting us to make the most of all the necessary services of the establishment.

We are additionally very grateful to our respected Vice Principal, HOD of Computer Science & Engineering, DSCE, Bangalore, Dr. Ramesh Babu D R, for his assist and encouragement.

We are immensely grateful to our respected and learned guide, Dr. Shubha Bhat, Associate Professor CSE, DSCE for his or her valuable assist and steerage. We are extremely thankful to them for all the encouragement and steerage they have given us during every stage of the project.

We want to thank our project co-ordinators Dr. Vindhya M, Associate Professor, CSE, DSCE for their guidance and support.

We are also grateful to all the other faculty and employees members of our department for their kind co-operation and help.

Lastly, we wish to express our deep appreciation in direction of our classmates and our household for providing us with fixed ethical help and encouragement.

  • SHALINI [1DS15CS093]

Facial Expression Recognition using Neural Networks


Current Human Machine Interaction (HMI) frameworks presently can’t seem to attain the total passionate and social abilities fundamental for rich and powerful connection with people to characterize faces in a given single picture or succession of images as one of the six important emotions. typical AI methodologies, for example, booster vector machines and Bayesian classifiers, have been effective whereas ordering introduced outward appearances in a managed area, late investigations have demonstrated that these preparations don’t have the adaptability to group footage caught in an unconstrained uncontrolled method (“in the wild”) or when connected databases for which they have been not structured. This poor generalizability of those strategies is essentially because of the method in which that quite a few methodologies are topic or database needy and just match for perceiving misrepresented or constrained articulations like these in the preparation database. In addition, acquiring accurate training data is particularly tough, particularly for emotions such as anger or unhappy that are very tough to accurately replicate.

Recently, because of an increase within the prepared availability of computational power and increasingly massive training databases to work with, the machine studying strategy of neural networks has seen resurgence in recognition. Recent state of the art outcomes have been obtained utilizing neural net-works within the fields of visual object recognition, human pose estimation, face verification and many more. Even in the FER area outcomes thus far have been promising. Unlike traditional machine learning

approaches where features are outlined by hand, we often see enchancment in visual processing tasks when using neural networks because of the network’s capacity to extract undefined options from the training database. It is often the case that neural networks which are educated on large amounts of knowledge are in a place to extract features generalizing nicely to scenarios that the network has not been trained on.

We discover this idea carefully by coaching our proposed network architecture on a subset of the available training databases, after which per-forming cross-database experiments which allow us to precisely judge the network’s efficiency in novel scenarios.

In the FER drawback, nonetheless, in distinction to visible object databases similar to imageNet, present FER databases of-ten have restricted numbers of topics, few pattern images or movies per expression, or small variation between sets, making neural networks considerably more difficult to coach. For example, the FER2013 database (one of the biggest lately released FER databases) contains 35,887 photographs of different subjects yet only 547 of the photographs portray disgust. Similarly, the CMU MultiPIE face database contains round 750,000 photographs however it is comprised of solely 337 different topics, the place 348,000 pictures portray solely a “neutral” emotion and the remaining images portray anger, concern or disappointment respectively. Dept. of CSE, DSCE, Bangalore 78 1Facial Expression Recognition utilizing Neural Networks


Problem Statement

Human facial expressions may be simply categorised into 7 basic feelings: pleased, unhappy, surprise, concern, anger and impartial. Facial feelings are expressed through theactivation of particular units of facial muscular tissues. These sometimes subtle, but advanced, signals in an expression usually contain an ample quantity of information about our state of mind. Through facial emotion recognition, we are ready to measure the effects that content material and providers have on the customers through an easyand low-cost process. For example, retailers might use there metrics to evaluate buyer interest. Healthcare suppliers can provide better service by using extra details about patients’ emotional state during the remedy. Humans are well-trained in studying the feelings of others, in fact, at simply 14 months old, infants can already inform the difference between happy and sad. We designed a deep learning neural network that provides machines the ability to make inferences about our emotional states.

Facial expression recognition is a course of performed by computers, which consists of:

  1. Detection of the face when the consumer comes within the internet cam’s frame.
  2. Extracting facial features from the detected face area and detecting the form of facial parts or describing the feel of the pores and skin in a facial space. This is identified as Facial Features Extraction.
  3. After the feature extraction, the computer categorizes the emotion states of the person through the datasets offered during coaching of the model. Dept. of CSE, DSCE, Bangalore seventy eight 2

Facial Expression Recognition using Neural Networks3 LITERATURE SURVEY

Literature Survey

Human Facial Expression Recognition from Static Image using Shape and Appearance Feature

Authors: Naveen Kumar , H N Jagadeesha , S Amith Kjain.


This paper proposes a Facial Expression Recognition using Histogram of Oriented Gradients (HOG) and Support Vector Machine(SVM). The proposed work exhibits how HOG features may be exploited for facial features recognition. Use of HOG features make the efficiency of FER system to be topic independent.

The accuracy of the work is found be ninety two.56% when carried out using Cohnkanade dataset for six primary expressions. Results indicate that form options on face carry more information for emotion modelling compared with texture and geometric options. Shape features are higher when compared with geometric features as a end result of the fact that a small pose variation degrades the efficiency of FER system which depends on geometric features the place as a small pose variation doesn’t replicate any changes on a FER system which depends on HOG features. Detection charges for disgust, concern and sad is much less within the proposed work. Detection rates could be additional improved by combining form, texture and geometric options. Optimized cell sizes could additionally be thought of for actual time implementation in order to address both detection fee and processing velocity. The affect of non-frontal face on the performance of FER system might be addressed in the future work.

Face Detection and Recognition utilizing Viola-Jones algorithm and Fusion of PCA and ANN

Authors : Narayan T. Deshpande , Dr. S. Ravishankar,

Description :

This paper suggest to Face recognition, Principal Component Analysis, Artificial Neural Network, Viola-Jones algorithm. The paper presents an efficient approach for face detection and recognition using Viola-Jones, fusion of PCA and ANN strategies. The performance of the proposed methodology is compared with other present face recognition strategies and it’s observed that better accuracy in recognition is achieved with the proposed method. Face detection and recognition plays an important role in a variety of functions. In many of the purposes a excessive rate of accuracy in figuring out an individual is desired therefore the proposed method can be considered compared with the existing strategies. Dept. of CSE, DSCE, Bangalore 78 3

Facial Expression Recognition using Neural Networks


Facial Expression Recognition

Authors : Neeta Sarode , Prof. Shalini Bhatia


This paper suggest to grayscale images; face; facial expression recognition; lip region extraction; human-computer interplay. Experiments are carried out on grayscale picture databases. Images from Yale facial image database and JAFFE database (Figure 7) are used for experiments. JAFFE database consists of grayscale photographs. The database consists of Japanese Female Facial Expressions that have 7 expressions of 10 people including neutral. Each particular person has 3-4 photographs of same expression, so the whole number of images within the database comes to 213 photographs.

An efficient, native image- based mostly method for extraction of intransient facial options and recognition of four facial expressions was introduced. In the face, we use the eyebrow and mouth corners as main ‘anchor’ factors. It does not require any manual intervention (like preliminary guide project of function points). The system, primarily based on an area approach, is ready to detect partial occlusions also.

Comparision of PCA and LDA Techniques for Face

Recognition Feature Based Extraction With Accuracy


Authors : Riddhi A. Vyas , Dr.S.M. Shah


This paper propose to Face recognition, PCA, LDA, Eigen value, Covariance, Euclidean distance, Eigen face, Scatter matrix. A characteristic extraction is a quite tricky part in a means of Recognition. To get higher fee of face recognition the correct alternative of algorithm from many for characteristic extraction is extremely important and that performs significant function in face recognition course of. Before selecting the feature extraction strategies you must have information of it and which one performs precisely in which standards. In this comparative analysis, it’s provided which Feature extraction technique is performs correct in numerous criteria.

From particular person conclusion it’s clear and proves that LDA is efficient for facial recognition method for photographs of Yale database, comparative examine point out that LDA achieved seventy four.47% recognition price with coaching set of sixty eight photographs and out of 165 pictures whole 123 photographs are acknowledged with larger accuracy. In future Face Recognition fee could be improved that features the complete frontal face with facial expression utilizing PCA and LDA. Face recognition Rate can be improved with hybrid preprocessing method for PCA and LDA. Both feature extraction approach can not give glad recognition rate for Illumination downside so it could beimproved. PCA and LDA can be combining with different strategies DWT, DCT,

LBP and so forth can improve the face recognition fee. Dept. of CSE, DSCE, Bangalore 78 4

Facial Expression Recognition using Neural Networks3 LITERATURE SURVEY

Facial Expression Recognition Using Visual Saliency and Deep Learning

Authors : Viraj Mavani , Shanmuganathan Raman , Krishna Prasad Miyapuram.


This paper propose to Facial Expression Recognition Using Visual Saliency and Deep Learning. We have demonstrated a CNN for facial features recognition with generalization skills. We tested the contribution of potential facial regions of curiosity in human vision utilizing visible saliency of images in our facial expressions datasets.The confusion between completely different facial expressions was minimal with high recognition accuracies for 4 emotions – disgust, joyful, unhappy and surprise [Table 1, 2]. The general human tendency of indignant being confused as sad was observed [Table 1] as given in [22]. Fearful was confused with impartial, whereas neutral was confused with sad. When saliency maps were used, we observed a change within the confusion matrix of emotion recognition accuracies. Angry, impartial and sad feelings were now more confused with disgust, whereas surprised was more confused as fearful [Table 2]. These outcomes instructed that the generalization of deep studying community with visual saliency sixty five.39% was much larger than chance degree of 1/7. Yet, the construction of confusion matrix was much totally different when in comparison with the deep learning community that thought of complete images. We conclude with the key contributions of the paper as two-fold. (i), we now have presented generalization of deep learning network for facial emotion recognition throughout two datasets. (ii), we introduce right here the concept of visual saliency of images as input and observe the habits of the deep studying network to be diversified. This opens up an exciting discussion on further integration of human emotion recognition (exemplified using visible saliency on this paper) and those of deep convolutional neural networks for facial features recognition. Dept. of CSE, DSCE, Bangalore seventy eight 5


Written by Essay Examples

A Portrait of the Artist as a Young Man

A Problem of Adolescent Rebellion