The repo for "Balanced Multimodal Learning via On-the-fly Gradient Modulation", CVPR 2022 (ORAL)
MIT License
236
stars
18
forks
source link
Can you help me analyze the reason why my training failed when fusion_method='gated ', but my testing and evaluation failed when fusion_method='concat' #46
Hello, thank you very much for your excellence work. But there are two questions I need your help to analyze the reasons for:
(1)I encountered the following error while using the CREMAD dataset when fusionmethod='gated ':
--train --dataset CREMAD --modulation OGM --fusion_method gated --modulation_starts 20 --modulationends 80 --alpha 0.8
in train_epoch
out_v = (torch.mm(v, torch.transpose(model.module.fusion_module.fc_out.weight[:, weight_size // 2:], 0, 1))
RuntimeError: mat1 and mat2 shapes cannot be multiplied (64x512 and 256x6)
(2)But when fusionmethod='concat ', it can be trained normally
--train --dataset CREMAD --modulation OGM --fusion_method concat --modulation_starts 20 --modulationends 80 --alpha 0.8
The training was successful, but the following error occurred during the execution of the test and evaluation:
_--test --dataset CREMAD --modulation OGM --fusion_method concat --modulation_starts 20 --modulationends 80 --alpha 0.8
in valid
model.eval()
AttributeError: '_IncompatibleKeys' object has no attribute 'eval'
Trained model loaded!
Hello, thank you very much for your excellence work. But there are two questions I need your help to analyze the reasons for:
(1)I encountered the following error while using the CREMAD dataset when fusionmethod='gated ': --train --dataset CREMAD --modulation OGM --fusion_method gated --modulation_starts 20 --modulationends 80 --alpha 0.8
(2)But when fusionmethod='concat ', it can be trained normally --train --dataset CREMAD --modulation OGM --fusion_method concat --modulation_starts 20 --modulationends 80 --alpha 0.8
The training was successful, but the following error occurred during the execution of the test and evaluation: _--test --dataset CREMAD --modulation OGM --fusion_method concat --modulation_starts 20 --modulationends 80 --alpha 0.8