Missing input embeddings

[tzjg7293]

Testing out on TwoStreamNetwork but it seems that all the the paths to dev/test input embeddings mentioned in phoenix-2014t_s2t_ensemble.yaml are missing.

Where can I find these input embeddings?

[ChenYutongTHU]

Hi,

Please refer to Evaluation (Multi-source ensemble) You need to generate these embeddings by running

for stream in video keypoint joint
do
python -m torch.distributed.launch --nproc_per_node 1 --use_env extract_feature.py --config experiments/configs/TwoStream/${dataset}_s2t_${stream}.yaml --output_split dev,test
done

Then, you can get the ensemble's prediction by

python -m torch.distributed.launch --nproc_per_node 1 --use_env prediction.py --config experiments/configs/TwoStream/${dataset}_s2t_ensemble.yaml

Thanks!