anita-hu / MSAF

Offical implementation of paper "MSAF: Multimodal Split Attention Fusion"
MIT License
76 stars 9 forks source link
action-recognition cmu-mosei multimodal-deep-learning multimodal-emotion-recognition multimodal-learning multimodal-sentiment-analysis ntu-rgbd pytorch ravdess

MSAF: Multimodal Split Attention Fusion

arXiv PWC

Code for the paper MSAF: Multimodal Split Attention Fusion. This is our implementation of the MSAF module and the three MSAF-powered multimodal networks.

If you use this code, please cite our paper:

@misc{su2020msaf,
      title={MSAF: Multimodal Split Attention Fusion}, 
      author={Lang Su and Chuqing Hu and Guofa Li and Dongpu Cao},
      year={2020},
      eprint={2012.07175},
      archivePrefix={arXiv},
      primaryClass={cs.CV}
}

Installation

Clone this repo along with submodules

git clone --recurse-submodules https://github.com/anita-hu/MSAF.git

Install dependencies

Method 1: Using environment.yml (installs dependencies for all three datasets)

With this method, you can skip dependency installation steps from the dataset specific README files

conda env create -f environment.yml

Method 2: Without environment.yml

This code was developed with Python 3.6, PyTorch 1.7.1 in Ubuntu 20.04.

Usage