search

dannyFan-0201 / Micro-expression-recognition-using-dual-branch-3DCNN-network-with-novel-attention-mechanism

2 stars 0 forks source link

readme

[Micro-expression recognition using dual-branch 3DCNN network with novel attention mechanism]

Chun-Ting Fang, Tsung-Jung Liu

Abstract : Abstract—Micro-expressions refer to small and imperceptible changes in facial expressions displayed by humans in a very short period of time, but these changes contain rich emotional information. In this paper, we propose a shallow dual-branch 3D-CNN backbone network architecture in the first stage as preliminary temporal and spatial feature learning. At the same time, we have also enhanced and optimized the CAM (Channel Attention Module) within CBAM (Convolutional Block Attention Module) so that it can be better applied to the extraction of subtle changes in micro-expression faces. Then we use GRU (gated recurrent unit) and MSMH (multi-scale multi-head attention mechanism) as the second stage feature self-attention extraction. The facial action unit (AU) is used to cut out key areas where micro-expressions occur to enhance the learning effect of local features. In addition to testing on commonly used microexpression datasets, we also tested on lie detection datasets. A large number of experimental results show that this method can achieve very good results with relatively simple input and attention mechanisms.

Network Architecture

CMFNet

Environment

or see the requirements.txt

How to try

Download dataset (Most datasets require an application to download)

[SMIC] [SAMM] [CASME II] [CAS(ME)3] [Real-life deception detection Database]

Set dataset path

Edit in Dual-Branch 3DCNN+AU.py (set path in config)

output_folder ='./data/negative/training_frames' # This will be automatically generated.
negativepath = './data/negative/negative_video'
positivepath = './data/negative/positive_video'
surprisepath = './data/negative/surprise_video'

Parameter settings

excel_file_path = "/excel_file.xlsx"
df = pd.read_excel(excel_file_path, sheet_name='Sheet1')
df["Action Units"] = df["Action Units"].astype(str) #Convert Action Units data to string.
AU_CODE = [1, 2, 4, 5, 6, 7, 9, 10, 12, 14, 15, 17, 43] #Select the Action Units you want.

Run training


python Dual-Branch 3DCNN+AU.py
  1. Set the dataset path.
  2. Select the Action Units you want.
  3. Set path and parameter details in model.

Run testing

After training is completed, the output test results will be automatically calculated.
The UF1/UAR/ACC of each Subject will be listed in detail, as well as the final average score.

Contact

If you have any question, feel free to contact danny80351@gmail.com

Citation