State-of-the-Art Deep Learning scripts organized by models - easy to train and deploy with reproducible accuracy and performance on enterprise-grade infrastructure.
13.48k
stars
3.21k
forks
source link
[Help Me] How to fine-tune WaveGlow for ENGLISH to HINDI. Pretrained model mismatching #763
Traceback (most recent call last):
File "train.py", line 559, in <module>
main()
File "train.py", line 415, in main
args.amp, args.checkpoint_path, local_rank)
File "train.py", line 252, in load_checkpoint
torch.cuda.set_rng_state(checkpoint['cuda_rng_state_all'][device_id])
KeyError: 'cuda_rng_state_all'
Traceback (most recent call last):
File "train.py", line 559, in <module>
main()
File "train.py", line 415, in main
args.amp, args.checkpoint_path, local_rank)
File "train.py", line 248, in load_checkpoint
checkpoint = torch.load(filepath, map_location='cpu')
File "/opt/conda/lib/python3.6/site-packages/torch/serialization.py", line 580, in load
return _legacy_load(opened_file, map_location, pickle_module, **pickle_load_args)
File "/opt/conda/lib/python3.6/site-packages/torch/serialization.py", line 760, in _legacy_load
result = unpickler.load()
ModuleNotFoundError: No module named 'glow'
@GrzegorzKarchNV sir, and everyone, please help me to download right version WaveGlow pretrained model with finetuning command sir. Ex. --segment-length - 8000 --wn-channels - 256
MuruganR96
commented
3 years ago
1st Try: WaveGlow parameters --segment-length - 8000 --wn-channels - 256
WaveGlow Model: https://ngc.nvidia.com/catalog/models/nvidia:waveglow_ckpt_amp_256/files
WaveGlow FP16 Single GPU:
2nd Try: WaveGlow parameters --segment-length - 16000 --wn-channels - 256
_waveglow_1076430_14000amp
wget https://api.ngc.nvidia.com/v2/models/nvidia/waveglow_ckpt_amp_256/versions/20.01.0/files/nvidia_waveglow256pyt_fp16.pt
3rd Try: WaveGlow parameters --segment-length - 8000 --wn-channels - 256
WaveGlow FP16 Single GPU:
4th Try: WaveGlow parameters --segment-length - 8000 --wn-channels - 256
_waveglow_256channels_ljsv3.pt
WaveGlow FP16 Single GPU:
python train.py -m WaveGlow -o ./ -lr 1e-4 --epochs 1001 -bs 10 --segment-length 8000 --weight-decay 0 --grad-clip-thresh 65504.0 --cudnn-enabled --cudnn-benchmark --log-file nvlog.json --training-files filelists/hindi_audio_text_train_filelist.txt --amp --validation-files filelists/hindi_audio_text_val_filelist.txt --wn-channels 256 --checkpoint-path backup/waveglow_256channels_ljs_v3.pt@GrzegorzKarchNV sir, and everyone, please help me to download right version WaveGlow pretrained model with finetuning command sir. Ex. --segment-length - 8000 --wn-channels - 256
Thanks Murugan Rajenthiran