liyhc
commented
2 months ago 这是因为iu.pth用的是iu-xray数据集训练配置，语言模型是Qwen不是Llama，另外各项参数得和iu.sh配置一样。
Open hitsz-zxw opened 2 months ago
hitsz-zxw
commented
2 months ago 十分感谢你的回答，现在我已经能够跑通iu_xray，还有一个问题，当我运行测试脚本iu.sh时，出现了以下报错
Traceback (most recent call last):
File "/disk0/Medical_Image_Analysis/R2GenCSR/train.py", line 55, in
然后我又尝试了运行你们提供的test_meteor.py，也是同样的报错BrokenPipeError，这似乎是Java评测脚本的问题，我该如何解决？或者我有什么方法可以得到模型生成的结果吗，这样我可以不通过Java库通过python的比如pycocoevalcap库来评测，以这样的方法能够复现你们论文中提到的结果吗？或者我通过逐个sample调用函数demo_test_step能得到相同的结果吗？
你好，很感谢你们做出的贡献，我对你们的工作很感兴趣，也跑通了你们提供的demo代码，但是，当我把模型从提供的mimic.pth换成iu.pth时，出现了以下错误： Loading vision encoder:/disk0/uni_group/Medical_Image_Analysis/R2GenCSR/VMamba/vssm_base_0229_ckpt_epoch_237.pth => merge config from ./VMamba/classification/configs/vssm1/vssm_base_224.yaml vssm => loaded pretrinaed vssm model successfully Loading Trainable vision encoder:/disk0/uni_group/Medical_Image_Analysis/R2GenCSR/VMamba/vssm_base_0229_ckpt_epoch_237.pth -- Done Loading LLAMA llama2 Loading checkpoint shards: 100%|███████████████████████████████████████████████████████| 2/2 [00:01<00:00, 1.31it/s] Loading LLAMA Done Traceback (most recent call last): File "/disk0/uni_group/Medical_Image_Analysis/R2GenCSR/demo.py", line 14, in
model = R2GenCSR(args)
File "/disk0/uni_group/ck/xiwen/Medical_Image_Analysis/R2GenCSR/models/R2GenCSR_domo.py", line 190, in init
self.load_state_dict(state_dict=state_dict, strict=False)
File "/disk0/uni_group/miniconda3/envs/R2GenCSR/lib/python3.10/site-packages/torch/nn/modules/module.py", line 2153, in load_state_dict
raise RuntimeError('Error(s) in loading state_dict for {}:\n\t{}'.format(
RuntimeError: Error(s) in loading state_dict for R2GenCSR:
size mismatch for llama_proj.weight: copying a param with shape torch.Size([2048, 1024]) from checkpoint, the shape in current model is torch.Size([4096, 1024]).
size mismatch for llama_proj.bias: copying a param with shape torch.Size([2048]) from checkpoint, the shape in current model is torch.Size([4096]).
size mismatch for layer_norm.weight: copying a param with shape torch.Size([2048]) from checkpoint, the shape in current model is torch.Size([4096]).
size mismatch for layer_norm.bias: copying a param with shape torch.Size([2048]) from checkpoint, the shape in current model is torch.Size([4096]).