search

microsoft / unilm

Large-scale Self-supervised Pre-training Across Tasks, Languages, and Modalities
https://aka.ms/GeneralAI
MIT License
20.21k stars 2.55k forks source link

How to do inference in downstream task ER s3prl wavlm #704

Closed sciai-ai closed 2 years ago

sciai-ai commented 2 years ago

I have trained a downstream ER using wavlm in s3prl task where a checkpoint has been saved dev-best.ckp.. Can you please share the inference code that can be used to check the emotion for a given wav file.

I know there is this code to test but this assumes the data to be in the format for IEMOCAP rather than simply a wav file `data="your IEMOCAP root"

python3 run_downstream.py -m evaluate -t test -e checkpoint_fold1.ckpt -o args.expdir=tgt_dir/er/fold1,,config.downstream_expert.datarc.root=$data`

Can we use the saved checkpoint to something along the lines of the code below from HF.

I would want the model to be loaded once quit reloading it every time there is a new wav file input

image {

Sanyuan-Chen commented 2 years ago

Hi, we evaluated the WavLM model on ER task following the s3prl repo, and didn't conduct the inference with a given wav file. I guess you can modify the IEMOCAPDataset to consume the input wave file. However, this issue is more relevant to s3prl codes, and you may obtain the solution to this issue in s3prl repo.