search

NVIDIA / NeMo

A scalable generative AI framework built for researchers and developers working on Large Language Models, Multimodal, and Speech AI (Automatic Speech Recognition and Text-to-Speech)
https://docs.nvidia.com/nemo-framework/user-guide/latest/overview.html
Apache License 2.0
11.71k stars 2.45k forks source link

Model Conformer-CTC-Char import error from toolkit #2915

Closed vikneo2017 closed 2 years ago

vikneo2017 commented 3 years ago

I am used nemo_toolkit==r1.4.0 However, I have load model error model if I loaded it with

import nemo.collections.asr as nemo_asr asr_model = nemo_asr.models.EncDecCTCModelBPE.from_pretrained(model_name="stt_en_conformer_transducer_medium").

VahidooX commented 3 years ago

This model is a Transducer based model and you need to load it with EncDecRNNTBPEModel:

asr_model = nemo_asr.models.EncDecRNNTBPEModel.from_pretrained(model_name="stt_en_conformer_transducer_medium")

Or just use ASRModel which works for all ASR models:

asr_model = nemo_asr.models.ASRModel.from_pretrained(model_name="stt_en_conformer_transducer_medium")

vikneo2017 commented 3 years ago

This model is a Transducer based model and you need to load it with EncDecRNNTBPEModel:

asr_model = nemo_asr.models.EncDecRNNTBPEModel.from_pretrained(model_name="stt_en_conformer_transducer_medium")

Or just use ASRModel which works for all ASR models:

asr_model = nemo_asr.models.ASRModel.from_pretrained(model_name="stt_en_conformer_transducer_medium")

Please, in which version of NeMo can you import models this way? I have tried NeMo r1.4.0, but it doesn't work

titu1994 commented 3 years ago

Nemo 1.4 is not released yet. You can use r1.4.0 branch or the main branch, or wait till next week for the release of 1.4 (we try to release on the first week of the month).

If that still doesn't work please post the full stack trace

vikneo2017 commented 3 years ago

Nemo 1.4 is not released yet. You can use main branch, or wait till next week for the release of 1.4 (we try to release on the first week of the month)

That is, while the possibility of loading a pre-trained model 'stt_en_conformer_transducer_medium' is not possible? Because in the main branch there is also no way to load the model through the class EncDecRNNTBPEModel....

titu1994 commented 3 years ago

The checkpoint exists and is public, the main branch and r1.4.0 branch code will be able to download and instantiate the RNNT models. What exactly is the error you are seeing ? Can you post the error logs ?

vikneo2017 commented 3 years ago

asr_model = nemo_asr.models.EncDecRNNTBPEModel.from_pretrained("stt_en_conformer_transducer_medium")

FileNotFoundError Traceback (most recent call last) /tmp/ipykernel_15020/737701909.py in ----> 1 asr_model = nemo_asr.models.EncDecRNNTBPEModel.from_pretrained("stt_en_conformer_transducer_medium")

~/stt-conf/conformer/lib/python3.8/site-packages/nemo/core/classes/common.py in from_pretrained(cls, model_name, refresh_cache, override_config_path, map_location, strict, return_config) 640 cls, 641 model_name: str, --> 642 refresh_cache: bool = False, 643 override_config_path: Optional[str] = None, 644 map_location: Optional['torch.device'] = None,

FileNotFoundError: Model stt_en_conformer_transducer_medium was not found. Check cls.list_available_models() for the list of all available models.

vikneo2017 commented 3 years ago

The checkpoint exists and is public, the main branch and r1.4.0 branch code will be able to download and instantiate the RNNT models. What exactly is the error you are seeing ? Can you post the error logs ?

If I load model from checkpoint:

asr_model = nemo_asr.models.EncDecRNNTBPEModel.restore_from("/home/user/CONFORMER/stt_en_conformer_transducer_medium.nemo")

ImportError Traceback (most recent call last) /tmp/ipykernel_15020/3187540524.py in ----> 1 asr_model = nemo_asr.models.EncDecRNNTBPEModel.restore_from("/home/user/CONFORMER/stt_en_conformer_transducer_medium.nemo")

~/stt-conf/conformer/lib/python3.8/site-packages/nemo/core/classes/modelPT.py in restore_from(cls, restore_path, override_config_path, map_location, strict, return_config) 417 418 else: --> 419 # Preserve the configuration 420 if not isinstance(optim_config, DictConfig): 421 optim_config = OmegaConf.create(optim_config)

~/stt-conf/conformer/lib/python3.8/site-packages/nemo/core/classes/modelPT.py in _default_restore_from(cls, restore_path, override_config_path, map_location, strict, return_config) 370 (Optionally) Setups data loader to be used in test, with support for multiple data loaders. 371 --> 372 Args: 373 test_data_layer_config: test data layer parameters. 374 """

~/stt-conf/conformer/lib/python3.8/site-packages/nemo/core/classes/common.py in from_config_dict(cls, config) 463 imported_cls_tb = traceback.format_exc() 464 --> 465 # try instantiating model with target class 466 if imported_cls is not None: 467 # if calling class (cls) is subclass of imported class,

~/stt-conf/conformer/lib/python3.8/site-packages/nemo/collections/asr/models/rnnt_bpe_models.py in init(self, cfg, trainer) 55 description="For details about this model, please visit https://ngc.nvidia.com/catalog/models/nvidia:nemo:stt_en_contextnet_1024", 56 location="https://api.ngc.nvidia.com/v2/models/nvidia/nemo/stt_en_contextnet_1024/versions/1.0.0/files/stt_en_contextnet_1024.nemo", ---> 57 ) 58 results.append(model) 59

ImportError: Could not import warprnnt_pytorch. Please visit https://github.com/HawkAaron/warp-transducer and follow the steps in the readme to build and install the pytorch bindings for RNNT Loss, or use the provided docker container that supports RNN-T loss.

VahidooX commented 3 years ago

Those two lines works for me fine. When was the last time you updated (pulled) your r1.4.0 branch? You may be using an outdated branch. Do you see the name of the model in this file? nemo/collections/asr/models/rnnt_bpe_models.py