uable to FINETUNE from a PRETRAINED model

[mujhenahiata]

❓

NEED HELP/FIX ASAP already logged issues.

3683 : https://github.com/facebookresearch/fairseq/issues/3683

AssertionError: Could not infer task type from {'_name': 'temp_sampled_audio_pretraining', 'data': '/home/sibadatta/asr/AI4BHARAT/IndicWav2Vec/data/hindi/manifest', 'max_sample_size': 320000, 'min_sample_size': 32000, 'normalize': True, 'sampling_alpha': 0.7}. Available argparse tasks: dict_keys(['hubert_pretraining', 'audio_pretraining', 'audio_finetuning', 'sentence_ranking', 'translation_multi_simple_epoch', 'cross_lingual_lm', 'speech_unit_modeling', 'translation', 'multilingual_translation', 'speech_to_text', 'text_to_speech', 'frm_text_to_speech', 'translation_lev', 'translation_from_pretrained_xlm', 'sentence_prediction', 'simul_speech_to_text', 'simul_text_to_text', 'online_backtranslation', 'multilingual_masked_lm', 'language_modeling', 'legacy_masked_lm', 'multilingual_language_modeling', 'sentence_prediction_adapters', 'semisupervised_translation', 'denoising', 'masked_lm', 'translation_from_pretrained_bart', 'multilingual_denoising', 'speech_to_speech', 'dummy_lm', 'dummy_masked_lm', 'dummy_mt']). Available hydra tasks: dict_keys(['hubert_pretraining', 'audio_pretraining', 'audio_finetuning', 'speech_unit_modeling', 'translation', 'translation_lev', 'translation_from_pretrained_xlm', 'sentence_prediction', 'simul_text_to_text', 'language_modeling', 'multilingual_language_modeling', 'sentence_prediction_adapters', 'masked_lm', 'dummy_lm', 'dummy_masked_lm'])

What have you tried?

I created the manifest as prescribed and used used this line

---

fairseq-hydra-train \ task.data=/path/to/finetune/manifest/directory/for/a/particular/language \ common.wandb_project= \ model.w2v_path=/path/to/pretrained/model_large.pt \ common.log_interval=50 \ common.log_format=tqdm \ dataset.max_tokens=1000000 \ checkpoint.save_dir=/path/to/save/model/fine_tune_checkpoints \ +optimization.update_freq='[1]' \ --config-dir /path/to/configs/directory \ --config-name ai4b_xlsr"

---

What's your environment?

and just to test the code, i wanted to run it on CPU

[gmryu]

What is this temp_sampled_audio_pretraining task? A task: _name: is not a "name" for your project. It refers a defined task class.

The reason https://github.com/facebookresearch/fairseq/issues/3683 happens is he did not import https://github.com/facebookresearch/fairseq/blob/main/examples/wav2vec/unsupervised/tasks/unpaired_audio_text.py during runtime. So python cannot find such task class.

There are two ways to import new .py for fairseq.

1. as https://github.com/facebookresearch/fairseq/issues/3683#issuecomment-877774572 stated, if the new defined .py is moved under fairseq/fairseq/{corresponding folder}, i.e. data task, they are automatically imported.
2. the other way is adding common.user_dir={a directory where you hold fairseq extension codes and folders}, for that issue it is ${FAIRSEQ_ROOT}/examples/wav2vec/unsupervised

So if you implemented a temp_sampled_audio_pretraining task class or .py. You need to import it by using user_dir. See examples/wav2vec/unsupervised, and you will know user_dir's required directory structure. If you can use an existed task class instead, I would recommend use that first. Hope this helps.