I use contextNet to train on the data librispeech. The model can be trained successfully, however, it returns error when testing. I observe a similar bug in https://github.com/openspeech-team/openspeech/issues/37, but it seems not be fixed?
AttributeError: 'ContextNetModel' object has no attribute 'optimizer'
The full log for this error is :
Testing DataLoader 0: 0%| | 0/174 [00:00<?, ?it/s]Er
ror executing job with overrides: ['dataset=librispeech', 'dataset.dataset_download=False', 'dataset.dataset_pat
h=/home/mnt/cftao/openspeech/data', 'dataset.manifest_file_path=/home/mnt/cftao/openspeech/manifest/contextnet-l
ibrispeech-0.txt', 'tokenizer=libri_subword', 'model=contextnet', 'audio=fbank', 'lr_scheduler=warmup_reduce_lr_
on_plateau', 'trainer=gpu', 'criterion=ctc']
Traceback (most recent call last):
File "/home/mnt/cftao/openspeech/./openspeech_cli/hydra_train.py", line 56, in hydra_main
trainer.test(model, data_module)
File "/home/mnt/cftao/anaconda3/envs/speech/lib/python3.9/site-packages/pytorch_lightning/trainer/trainer.py",
line 868, in test
return self._call_and_handle_interrupt(self._test_impl, model, dataloaders, ckpt_path, verbose, datamodule)
File "/home/mnt/cftao/anaconda3/envs/speech/lib/python3.9/site-packages/pytorch_lightning/trainer/trainer.py",
line 652, in _call_and_handle_interrupt
return self.strategy.launcher.launch(trainer_fn, *args, trainer=self, **kwargs)
File "/home/mnt/cftao/anaconda3/envs/speech/lib/python3.9/site-packages/pytorch_lightning/strategies/launchers
/multiprocessing.py", line 103, in launch
mp.start_processes(
File "/home/mnt/cftao/anaconda3/envs/speech/lib/python3.9/site-packages/torch/multiprocessing/spawn.py", line
197, in start_processes
while not context.join():
File "/home/mnt/cftao/anaconda3/envs/speech/lib/python3.9/site-packages/torch/multiprocessing/spawn.py", line
160, in join
raise ProcessRaisedException(msg, error_index, failed_process.pid)
torch.multiprocessing.spawn.ProcessRaisedException:
-- Process 1 terminated with the following error:
Traceback (most recent call last):
File "/home/mnt/cftao/anaconda3/envs/speech/lib/python3.9/site-packages/torch/multiprocessing/spawn.py", line
69, in _wrap
fn(i, *args)
File "/home/mnt/cftao/anaconda3/envs/speech/lib/python3.9/site-packages/pytorch_lightning/strategies/launchers
/multiprocessing.py", line 129, in _wrapping_function
results = function(*args, **kwargs)
File "/home/mnt/cftao/anaconda3/envs/speech/lib/python3.9/site-packages/pytorch_lightning/trainer/trainer.py",
line 915, in _test_impl
results = self._run(model, ckpt_path=self.ckpt_path)
File "/home/mnt/cftao/anaconda3/envs/speech/lib/python3.9/site-packages/pytorch_lightning/trainer/trainer.py",
line 1166, in _run
results = self._run_stage()
File "/home/mnt/cftao/anaconda3/envs/speech/lib/python3.9/site-packages/pytorch_lightning/trainer/trainer.py",
line 1249, in _run_stage
return self._run_evaluate()
File "/home/mnt/cftao/anaconda3/envs/speech/lib/python3.9/site-packages/pytorch_lightning/trainer/trainer.py",
line 1294, in _run_evaluate
eval_loop_results = self._evaluation_loop.run()
File "/home/mnt/cftao/anaconda3/envs/speech/lib/python3.9/site-packages/pytorch_lightning/loops/loop.py", line
200, in run
self.advance(*args, **kwargs)
File "/home/mnt/cftao/anaconda3/envs/speech/lib/python3.9/site-packages/pytorch_lightning/loops/dataloader/eva
luation_loop.py", line 155, in advance
dl_outputs = self.epoch_loop.run(self._data_fetcher, dl_max_batches, kwargs)
File "/home/mnt/cftao/anaconda3/envs/speech/lib/python3.9/site-packages/pytorch_lightning/loops/loop.py", line
200, in run
self.advance(*args, **kwargs)
File "/home/mnt/cftao/anaconda3/envs/speech/lib/python3.9/site-packages/pytorch_lightning/loops/epoch/evaluati
on_epoch_loop.py", line 143, in advance
output = self._evaluation_step(**kwargs)
File "/home/mnt/cftao/anaconda3/envs/speech/lib/python3.9/site-packages/pytorch_lightning/loops/epoch/evaluati
on_epoch_loop.py", line 240, in _evaluation_step
output = self.trainer._call_strategy_hook(hook_name, *kwargs.values())
File "/home/mnt/cftao/anaconda3/envs/speech/lib/python3.9/site-packages/pytorch_lightning/trainer/trainer.py",
line 1703, in _call_strategy_hook
output = fn(*args, **kwargs)
File "/home/mnt/cftao/anaconda3/envs/speech/lib/python3.9/site-packages/pytorch_lightning/strategies/ddp_spawn
.py", line 300, in test_step
return self.model.test_step(*args, **kwargs)
File "/home/mnt/cftao/openspeech/openspeech/models/contextnet/model.py", line 155, in test_step
return self.collect_outputs(
File "/home/mnt/cftao/openspeech/openspeech/models/openspeech_ctc_model.py", line 89, in collect_outputs
"learning_rate": self.get_lr(),
File "/home/mnt/cftao/openspeech/openspeech/models/openspeech_model.py", line 211, in get_lr
for g in self.optimizer.param_groups:
File "/home/mnt/cftao/anaconda3/envs/speech/lib/python3.9/site-packages/torch/nn/modules/module.py", line 1614
, in __getattr__
raise AttributeError("'{}' object has no attribute '{}'".format(
AttributeError: 'ContextNetModel' object has no attribute 'optimizer'
I also try to switch the model from contextnet to contextnet_lstm, the problem exists.
Environment info
Information
I use contextNet to train on the data librispeech. The model can be trained successfully, however, it returns error when testing. I observe a similar bug in https://github.com/openspeech-team/openspeech/issues/37, but it seems not be fixed?
AttributeError: 'ContextNetModel' object has no attribute 'optimizer'The full log for this error is :
I also try to switch the model from contextnet to contextnet_lstm, the problem exists.
Thanks for your time!