MEAN architecture
Characteristics:
- Shallow network
- Multi-stream
- Multi-output with two task-specific networks
- Capable of both micro-expression spotting and recognition
Results
Take note that analysis is equivalent to spot-then-recognize and STRS is our proposed evaluation metric. Please refer to the paper for more experiment results. Here is the detailed result for micro-expression spotting (S) and analysis (A):
How to run the code
Step 1) Download the micro-expression datasets for experiment, we suggest the files to be structured as follows:
├─MEAN_Weights
├─Utils
├─define_model.py
├─face_crop.py
├─feature_extraction.py
├─load_excel.py
├─load_images.py
├─main.py
├─prepare_training.py
├─train_evaluate.py
├─training_utils.py
├─requirements.txt
├─CASME_sq├─code_final.xlsx
├─rawpic├─CASME2
├─CASME2
├─CASME2-RAW
└─CASME2_label_Ver_2.xls├─SAMM
└─SAMM_20181215_Micro_FACS_Codes_v2.xlsx
├─SAMMLV
├─SAMM_longvideos
└─SAMM_LongVideos_V2_Release.xlsx└─SMIC
├─SMIC-E_raw image
├─HS_long├─SMIC-HS-E
├─HS
└─SMIC-HS-E_annotation.xlsx├─NIR_long
└─SMIC-NIR-E
├─NIR
└─SMIC-NIR-E_annotation.xlsx└─VIS_long
└─SMIC-VIS-E
├─VIS
└─SMIC-VIS-E_annotation.xlsx
Step 2) Installation of packages using pip
pip install -r requirements.txt
Step 3) MEAN Training and Evaluation
python main.py
Note for parameter settings
--dataset_name (CASME_sq/SAMMLV/CASME2/SMIC_HS/SMIC_VIS/SMIC_NIR)
--train (True/False)
Additional Notes
If you find this work useful, please cite the paper:
@article{liong2023spot,
title={Spot-then-Recognize: A Micro-Expression Analysis Network for Seamless Evaluation of Long Videos},
author={Liong, Gen-Bing and See, John and Chan, Chee-Seng},
journal={Signal Processing: Image Communication},
volume={110},
pages={116875},
year={2023},
publisher={Elsevier}
}