关于MMSA_test 无法正常使用

Ben5460 commented 1 year ago

你好！我成功地用你们的MMSA_run 训练出来了misa mosei 在models文件夹下的模型，也用你们的fes 提出来新视频的特征。但是我运行mmsa_test时运行到最后报错。这是我训练时用的命令 python -m MMSA -d mosei -m misa --model-save-dir ./models --res-save-dir ./results

这是我的测试脚本 _from MMSA.config import get_config_tune from MMSA import MMSA_test model = "/home/ben/hdd/AI/misa.pth" fea= "/home/ben/hdd/AI/feature.pkl" config = get_config_tune('misa', 'mosei') MMSA_test(config,model,fea,0)__

报错内容 Traceback (most recent call last): File "/home/ben/hdd/MMSA-master/run.py", line 8, in MMSA_test(config,model,fea,0) File "/home/ben/hdd/anaconda3/envs/AI/lib/python3.9/site-packages/MMSA/run.py", line 323, in MMSA_test model.load_state_dict(torch.load(weights_path), strict=False) File "/home/ben/hdd/anaconda3/envs/AI/lib/python3.9/site-packages/torch/nn/modules/module.py", line 1604, in load_state_dict raise RuntimeError('Error(s) in loading state_dict for {}:\n\t{}'.format( RuntimeError: Error(s) in loading state_dict for AMIO: size mismatch for Model.vrnn1.weight_ih_l0: copying a param with shape torch.Size([140, 35]) from checkpoint, the shape in current model is torch.Size([708, 177]). size mismatch for Model.vrnn1.weight_hh_l0: copying a param with shape torch.Size([140, 35]) from checkpoint, the shape in current model is torch.Size([708, 177]). size mismatch for Model.vrnn1.bias_ih_l0: copying a param with shape torch.Size([140]) from checkpoint, the shape in current model is torch.Size([708]). 。。

请问是我torch的版本不一致问题，还是mmsa_test函数config文件设置不正确，导致这个size不一样的问题？不知道能不能提供你们测试时的torch 版本，以及mmsa_test里所需要的config文件（如果我的用法错误），或者提供所有mmsa_test 里所需的参数文件

FlameSky-S commented 1 year ago

您好，测试和训练需要使用相同的特征提取方法。可以在训练是指定custom_feature参数：

MMSA_run(
    'tfr_net', 'mosei', config = config, gpu_ids = [2], seeds = [1],
    custom_feature="/home/sharing/mhs/features/feature_MOSEI.pkl",
    model_save_dir = "/home/sharing/mhs/MMSA/model_save",
    res_save_dir = "/home/sharing/mhs/MMSA/results",
    log_dir = "/home/sharing/mhs/MMSA/logs"
)

Ben5460 commented 1 year ago

非常感谢您和您的团队提供多模态情绪分析的工具以及解答！

我在对新视频进行测试上还是有几个问题：

如果我想使用MISA 和MOSEI 测试新视频应该怎么使用您的工具（流程）
您说测试和训练使用相同的特征提取方法是不是要和使用MMSA-FET保持一致
您提到的custom_feature里有feature_MOSEI.pkl 文件应该怎样获取，还是这个文件参数是我用MMSA-FET对新视频提取的特征文件

再次感谢您的宝贵时间。

FlameSky-S commented 1 year ago

先确定符合需求的特征提取方法；然后用该方法对一个数据集提取特征，得到自定义的特征文件；然后在该数据集上使用自定义特征进行训练调参；最后用相同的特征提取方法对测试视频进行特征提取，然后用训练好的模型进行测试

meng-alt commented 1 year ago

请问MMSA_test您运行成功了吗

randi-source commented 1 year ago

Hi everyone, I do have the same problem with the same error, i tried using MOSI dataset and my own dataset (with the help of MMSA_FET) to create model with MMSA_run. After the model is ready, i tried MMSA_test with MOSI dataset, my own custom dataset, and my own custom dataset with one video only, but this is still not working, can anyone pinpoint the case? Thank you for your time

zhanxingluan commented 5 months ago

Hi everyone, I do have the same problem with the same error, i tried using MOSI dataset and my own dataset (with the help of MMSA_FET) to create model with MMSA_run. After the model is ready, i tried MMSA_test with MOSI dataset, my own custom dataset, and my own custom dataset with one video only, but this is still not working, can anyone pinpoint the case? Thank you for your time

I have the same problem with you.Have you solved this problem?

haikunzhang95 commented 2 months ago

非常感谢您和您的团队提供多模态情绪分析的工具以及解答！

我在对新视频进行测试上还是有几个问题：

如果我想使用MISA 和MOSEI 测试新视频应该怎么使用您的工具（流程）

您说测试和训练使用相同的特征提取方法是不是要和使用MMSA-FET保持一致

您提到的custom_feature里有feature_MOSEI.pkl 文件应该怎样获取，还是这个文件参数是我用MMSA-FET对新视频提取的特征文件

再次感谢您的宝贵时间。

您好，我在使用MMSA_test()时候也有一些疑问，方便交流一些么？

thuiar / MMSA

关于MMSA_test 无法正常使用 #58