k2-fsa / snowfall

Moved to https://github.com/k2-fsa/icefall
Apache License 2.0
143 stars 42 forks source link

error when run "python3 ./mmi_bigram_train.py" #187

Closed yujunlhz closed 3 years ago

yujunlhz commented 3 years ago

after a few hours:

File "./mmi_bigram_train.py", line 502, in main() File "./mmi_bigram_train.py", line 451, in main save_checkpoint(filename=best_model_path, TypeError: save_checkpoint() missing 1 required positional argument: 'scaler'

ls -lt exp-lstm-adam-mmi-bigram-musan-dist/tensorboard/ | head -rw-r--r--. 1 root root 38056 May 7 22:15 events.out.tfevents.1620424437.c0d2246e17f0.697.0 drwxr-xr-x. 2 root root 136 May 7 22:01 train_weight_max_norms_linear.bias drwxr-xr-x. 2 root root 136 May 7 22:01 train_weight_max_norms_linear.weight drwxr-xr-x. 2 root root 136 May 7 22:01 train_weight_max_norms_lstms.4.bias_hh_l0 drwxr-xr-x. 2 root root 136 May 7 22:01 train_weight_max_norms_lstms.4.bias_ih_l0 drwxr-xr-x. 2 root root 136 May 7 22:01 train_weight_max_norms_lstms.4.weight_hh_l0 drwxr-xr-x. 2 root root 136 May 7 22:01 train_weight_max_norms_lstms.4.weight_ih_l0 drwxr-xr-x. 2 root root 136 May 7 22:01 train_weight_max_norms_lstms.3.bias_hh_l0 drwxr-xr-x. 2 root root 136 May 7 22:01 train_weight_max_norms_lstms.3.bias_ih_l0

I am using docker 10.2-cudnn7-devel-ubuntu18.04, Python 3.8.10, torch==1.8.1+cu102

Any ideas? Thanks

csukuangfj commented 3 years ago

https://github.com/k2-fsa/snowfall/blob/cd513af3cdefd8937fa42584f1c60baa722f9cb4/egs/librispeech/asr/simple_v1/mmi_bigram_train.py#L451-L460

Add a keyword argument

save_checkpoint(
...
scaler=None
)

You have to change other calls to save_checkpoint in the same file.

Sorry for the inconvenience.

danpovey commented 3 years ago

we probably should have given that a default value?

csukuangfj commented 3 years ago

https://github.com/k2-fsa/snowfall/blob/cd513af3cdefd8937fa42584f1c60baa722f9cb4/snowfall/common.py#L169

In general, people writes

 scaler: Optional[GradScaler] = None,

I agree to give it a default value.

csukuangfj commented 3 years ago

Closing via #190