search

NVIDIA / NeMo

A scalable generative AI framework built for researchers and developers working on Large Language Models, Multimodal, and Speech AI (Automatic Speech Recognition and Text-to-Speech)
https://docs.nvidia.com/nemo-framework/user-guide/latest/overview.html
Apache License 2.0
11.84k stars 2.46k forks source link

how to load and examine the pretrained models #1152

Closed niloofarmaani1 closed 3 years ago

niloofarmaani1 commented 4 years ago

hi I am trying to use a nemo asr pretrained model not for a new task, I just want to learnr how to load them and see the WER. I am using this tutorial: https://github.com/NVIDIA/NeMo/blob/main/tutorials/asr/01_ASR_with_NeMo.ipynb the part about getting out of box model: quartznet = nemo_asr.models.EncDecCTCModel.from_pretrained(model_name="QuartzNet15x5Base-En") how can I get other models? when I TRY jasper = nemo_asr.models.ASRConvCTCModel.from_p retrained(model_info='JasperNet10x5-En') it returns error: module 'nemo.collections.asr.models' has no attribute 'ASRConvCTCModel' how can I get the list of available pretrained models? and then after I get them I will use : params['model']['validation_ds']['batch_size'] = 16 first_asr_model.setup_test_data(test_data_config=params['model']['validation_ds']) first_asr_model.cuda() wer_nums = [] wer_denoms = [] for test_batch in first_asr_model.test_dataloader(): test_batch = [x.cuda() for x in test_batch] targets = test_batch[2] targets_lengths = test_batch[3]
log_probs, encoded_len, greedy_predictions = first_asr_model( input_signal=test_batch[0], input_signal_length=test_batch[1] )

    wer_num, wer_denom = first_asr_model._wer(greedy_predictions, targets, targets_lengths)
    wer_nums.append(wer_num.detach().cpu().numpy())
    wer_denoms.append(wer_denom.detach().cpu().numpy())
   print(f"WER = {sum(wer_nums)/sum(wer_denoms)}")

to examine them and see the wer, is that okay to do? what else should I do in order to make a pretrained model work? could someone please help with a complete example with the provided code and dataset I am very confused thanks

okuchaiev commented 4 years ago

"how can I get the list of available pretrained models?"

  • Use "list_available_models()" - only those models can be instantiated with "from_pretrained()" We'll be gradually adding more models to the selection.
viveksj commented 4 years ago

You can try to use a DeepPavlov wrapper to use the custom NeMo QuartzNet model. You could write your own DeepPavlov config file based on https://github.com/deepmipt/DeepPavlov/blob/master/deeppavlov/configs/nemo/asr.json.

Change load_path value with path to the directory containing your JasperEncoder and JasperDecoderForCTC checkpoints. nemo_params_path should be path to NeMo modules config file.

Note that this config for NeMo in DeepPavlov is slightly differs from default NeMo config. Necessary arguments could be taken from DeepPavlov apiref, but you could look at quartznet15x5.yaml from http://files.deeppavlov.ai/deeppavlov_data/nemo/quartznet15x5.tar.gz and modify your own config according to it.

^I am still trying to figure out how to do this, but the folks at Pavlov were kind enough to send me these instructions when I told them I was working on a NeMo model and was having difficulty using a model post training and found their wrapper Speech to Text NeMo integration.

samabdullah commented 4 years ago

i have the same problem? im just confused > can you get any update on your issue