GeWu-Lab / OGM-GE_CVPR2022

The repo for "Balanced Multimodal Learning via On-the-fly Gradient Modulation", CVPR 2022 (ORAL)
MIT License
236 stars 18 forks source link

CREMAD 数据集出现错误 #37

Open xwybupt opened 1 year ago

xwybupt commented 1 year ago

(RIS-MISO) xwy@xwy-Z390-AORUS-PRO:~/work/OGM-GE$ python main.py --train --dataset CREMAD --fusion_method gated --modulation OGM --modulation_starts 0 --modulation_ends 80 --alpha 0.3 --ckpt_path ckpt Namespace(alpha=0.3, audio_path='/home/xwy/work/OGM-GE/data/CREMA-D/AudioWAV', batch_size=64, ckpt_path='ckpt', dataset='CREMAD', epochs=100, fps=1, fusion_method='gated', gpu_ids='0', learning_rate=0.001, lr_decay_ratio=0.1, lr_decay_step=70, modulation='OGM', modulation_ends=80, modulation_starts=0, optimizer='sgd', random_seed=0, tensorboard_path=None, train=True, use_tensorboard=False, use_video_frames=3, visual_path='/home/xwy/work/OGM-GE/data/CREMA-D/') Epoch: 0: Start training ... Traceback (most recent call last): File "main.py", line 361, in main() File "main.py", line 301, in main train_dataloader, optimizer, scheduler) File "main.py", line 89, in train_epoch out_v = (torch.mm(v, torch.transpose(model.module.fusion_module.fc_out.weight[:, weight_size // 2:], 0, 1)) RuntimeError: mat1 and mat2 shapes cannot be multiplied (64x512 and 256x6)

echo0409 commented 1 year ago

Hi,

It seems that the dimension of model output does not match the setting in main.py (line 89). Please check the dimension of input data, each module of model, and setting in main.py are matched or not.

Thanks for your attention.

Felix-fz commented 2 months ago

我遇到了相同的问题,请问您解决了吗