search

SeanNaren / deepspeech.pytorch

Speech Recognition using DeepSpeech2.
MIT License
2.1k stars 620 forks source link

Not able to load the saved Checkpoints #654

Closed kanuarj closed 3 years ago

kanuarj commented 3 years ago

I have came across an error pertaining to load the checkpoints. My checkpoints are saved in /outputs/2021-06-23/17-59-11/lightning_logs/version_0/checkpoints/epoch=N-step=N.ckpt path. But when I try the command : python train.py trainer.resume_from_checkpoint=lightning_logs/deepspeech_checkpoint_epoch_N_iter_N.ckpt to load them I get an error. I solved the EOF error of name and also tried yet I get the error as 

/home/User/anaconda3/lib/python3.7/site-packages/pytorch_lightning/utilities/distributed.py:68: UserWarning: No checkpoint file exists at `resume_from_checkpoint`. Start from scratch

I even tried full path. Any suggestions ?

uygarkurt commented 3 years ago

hey I believe I guess terminal can't interpret the file name correctly. I encountered similar error during inference. Just rename the *.ckpt file and try again. By the way did you do any inference so far?

kanuarj commented 3 years ago

I did tried that. Yet it restarts from beginning. There was an issue with naming conventions while being saved with = sign which I did renamed manually and gave the path too.

stale[bot] commented 3 years ago

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

atifemreyuksel commented 2 years ago

@kanuarj and @uygarrr I encountered the same issue and solved by feeding the absolute path of checkpoint. In this repo, the example command is mistakenly written with relative path in readme but in pytorch lightning documentation they stated that checkpoint should be given as absolute path.

You can find the details in https://github.com/ashleve/lightning-hydra-template/blob/main/README.md.