Deep Learning Emotion Recognition in Group Photos
A deep learning web application that detects and classifies facial expressions in images. Built with PyTorch and Flask, featuring a ResNet-18 model trained on the RAF-DB dataset achieving 80% accuracy.
A deep learning web app that detects and classifies facial expressions in images. Built with PyTorch and Flask, using a ResNet-18 model trained on the RAF-DB dataset at 80% accuracy.
Read the full technical deep-dive →
I used a two-stage approach with MTCNN as the primary face detector and Haar Cascade as fallback. MTCNN handles multiple faces in group photos well, and Haar Cascade picks up faces that MTCNN misses (unusual angles, extreme lighting).
I used transfer learning with ResNet-18 pretrained on ImageNet and fine-tuned on RAF-DB (Real-world Affective Faces Database), which has 15,339 images across 7 emotion classes. The custom classification head uses batch normalization and dropout for regularization.
RAF-DB has serious class imbalance (Happiness: 39%, Fear: 2.3%). I addressed this with focal loss, class weights, and weighted sampling to improve performance on minority classes like Fear and Disgust. The ensemble of models trained with different loss functions ended up being the most effective approach.
I built a Flask web app with image upload, real-time emotion detection, and annotated output showing bounding boxes and confidence scores. Deployed on Hugging Face Spaces using Docker.
| Component | Technology |
|---|---|
| Model | ResNet-18 (transfer learning from ImageNet) |
| Face Detection | MTCNN + Haar Cascade fallback |
| Backend | Flask + Gunicorn |
| Frontend | Vanilla JS with CSS animations |
| Dataset | RAF-DB (Real-world Affective Faces Database) |
| Deployment | Docker on Hugging Face Spaces |
| Metric | Value |
|---|---|
| Accuracy | 80% |
| Dataset | RAF-DB |
| Architecture | ResNet-18 |
| Input Size | 100x100 |
| Emotion | Performance | Notes |
|---|---|---|
| Happiness | Highest | Largest class in dataset |
| Neutral | High | Well-represented class |
| Surprise | Good | Distinctive facial features |
| Sadness | Moderate | Subtle expressions |
| Anger | Moderate | Often confused with intense expressions |
| Fear | Lower | Only 2.3% of training data |
| Disgust | Lower | Only 5% of training data |