search

pengfei-luo / MIMIC

[KDD 2023] Multi-Grained Multimodal Interaction Network for Entity Linking
16 stars 3 forks source link

cannot reproduce the RichpediaMEL results? #2

Closed zhiweihu1103 closed 1 year ago

zhiweihu1103 commented 1 year ago

Hi, Pengfei. Nice work. I find I cannot reproduce the RichpediaMEL dataset result,, I use the same yaml as you provided, can you help me? attachment is the training logs. richpediamel.txt

pengfei-luo commented 1 year ago

As you can see on Wandb, I confirm that there are no changes made to the code about model and data processing. We have verified that the pre-trained model, environment configuration, code, and data are all consistent. I don't think the CUDA version and NVIDIA driver version should have such a significant impact.

zhiweihu1103 commented 1 year ago

This is a very vexing question because I can't think of any other reason that could cause this problem.

pengfei-luo commented 1 year ago

My thought is, perhaps you can make some adjustments to the parameters and then observe if there's a decrease in loss within a few iterations (compared to your previous loss). Maybe consider replacing the optimizer with stochastic gradient descent? I'm not sure.

zhiweihu1103 commented 1 year ago

I'll try adjusting the parameters tomorrow and keep in touch at any time.

pengfei-luo commented 1 year ago

I discovered that the folder for the mention images should be named mention_images instead of mention_image. This would result in the absence of image data during the training process. This could also be the reason why you were unable to reproduce the original results.

pengfei-luo commented 1 year ago

You need to manually modify the following line in the config file for RichpediaMEL.

https://github.com/pengfei-luo/MIMIC/blob/59ef385c14c5bffd70eaf8012f876850f6b99072/config/richpediamel.yaml#L10

zhiweihu1103 commented 1 year ago

It is indeed like this, a very subtle problem, thank you very much, I will re-run the code and give the final result.

zhiweihu1103 commented 1 year ago

As a reminder, would you mind uploading the numerical results of Figure 4?

pengfei-luo commented 1 year ago

I have just updated the detailed results in the README file.

zhiweihu1103 commented 1 year ago

Great, Thx.

zhiweihu1103 commented 1 year ago

Hi, Pengfei. I have reproduced the results, thanks for your solution. Good luck. I will close the issue.

zhangzef commented 9 months ago

我似乎无法复现wikimel中的结果，我的超参数文件使用的是作者在GitHub中提供的文件，能麻烦您告诉我您在复现的过程中有什么需要注意的吗