Sxjdwang / TalkLip

373 stars 34 forks source link

AttributeError: 'AVHubertSeq2Seq' object has no attribute 'num_updates' #15

Open secnavo opened 1 year ago

secnavo commented 1 year ago

Run the training script according to readme.md, the error is as follows:

Traceback (most recent call last):
  File "train.py", line 740, in <module>
    train(device, {'gen': imGen, 'disc': imDisc}, avhubert, criterion, {'train': train_data_loader, 'test': test_data_loader},
  File "train.py", line 531, in train
    average_sync_loss, valid_log = eval_model(data_loader['test'], avhubert, criterion, global_step, device, model['gen'], model['disc'], args.cont_w, recon_loss)
  File "train.py", line 592, in eval_model
    lip_loss, sample_size, logs, enc_out = criterion(avhubert, sample)
  File "/data/anaconda3/envs/talklip/lib/python3.8/site-packages/torch/nn/modules/module.py", line 1130, in _call_impl
    return forward_call(*input, **kwargs)
  File "/data/ljy/TalkLip/av_hubert/fairseq/fairseq/criterions/label_smoothed_cross_entropy.py", line 79, in forward
    net_output = model(**sample["net_input"])
  File "/data/anaconda3/envs/talklip/lib/python3.8/site-packages/torch/nn/modules/module.py", line 1130, in _call_impl
    return forward_call(*input, **kwargs)
  File "/data/ljy/TalkLip/av_hubert/avhubert/hubert_asr.py", line 494, in forward
    ft = self.freeze_finetune_updates <= self.num_updates
  File "/data/anaconda3/envs/talklip/lib/python3.8/site-packages/torch/nn/modules/module.py", line 1207, in __getattr__
    raise AttributeError("'{}' object has no attribute '{}'".format(
AttributeError: 'AVHubertSeq2Seq' object has no attribute 'num_updates'

Can someone help to see where the problem is? Thanks! The code to run the script is as follows: python train.py --file_dir /data/wwp/dataset/LRS2 --video_root /data/wwp/dataset/LRS2/mvlrs_v1/main --audio_root /data/wwp/dataset/LRS2/valid_audio \ --bbx_root /data/wwp/dataset/LRS2/valid_bbx --word_root /data/wwp/dataset/LRS2/mvlrs_v1/main --avhubert_root ./av_hubert/avhubert --avhubert_path /data/ljy/checkpoints/TalkLip/lip_reading_expert.pt \ --checkpoint_dir ./checkpoints/ --log_name log_talklip_01 --n_epoch 10 --ckpt_interval 50

Sxjdwang commented 1 year ago

Sorry, I forgot to mention that some files in AV-hubert need to be modified. I have updated the readme file and uploaded two files to replace the existing ones in AV-hubert. Please refer to the train instruction in the readme for further details. Thank you.