ELIM_FER

Optimal Transport-based Identity Matching for Identity-invariant Facial Expression Recognition (NeurIPS 2022)

PAPER | DEMO

Daeha Kim, Byung Cheol Song

CVIP Lab, Inha University

Update

• 2022.09.20: Initialize this repository.

Requirements

• Python (>=3.8)
• PyTorch (>=1.7.1)
• pretrainedmodels (>=0.7.4)
• Wandb
• Fabulous (terminal color toolkit)

To install all dependencies, do this.

pip install -r requirements.txt

Datasets

1. Download four public benchmarks for training and evaluation (please download after agreement accepted).

   • AffectNet
   • Aff-wild
   • Aff-wild2
   • AFEW-VA

   (For more details visit website)

2. Follow preprocessing rules for each dataset by referring pytorch official custom dataset tutorial.

Training

Just run the below script!

chmod 755 run.sh
./run.sh <method> <gpu_no> <port_no> 

• <method>: elim or elim_category
• <gpu_no>: GPU number such as 0 (or 0, 1 etc.)
• <port_no>: port number to clarify workers (e.g., 12345)
• Note: If you want to try 7-class task (e.g., AffectNet), add age_script folder to your train or val. script and turn on elim_category option.

Evaluation

• Evaluation is performed automatically at each print_check point in training phase.

Demo

• Do to demo folder, and then feel free to use.
• Real-time demo with pre-trained weights

  

TODO

• [x] Refactoring
• [ ] Upload pre-trained model weights
• [x] Upload demo files
• [x] Upload train/eval files

Note

• In case of Mlp-Mixer, please refer official repository (link)

Citation

If our work is useful for your work, then please consider citing below bibtex:

@misc{kim2022elim,
    author = {Kim, Daeha and Song, Byung Cheol},
    title = {Optimal Transport-based Identity Matching for Identity-invariant Facial Expression Recognition},
    Year = {2022},
    Eprint = {arXiv:2209.12172}
}

Contact

If you have any questions, feel free to contact me at kdhht5022@gmail.com.