search

er-muyue / DeFRCN

MIT License
181 stars 43 forks source link

Unable to reproduce the results #35

Open fenghuo-cao opened 2 years ago

fenghuo-cao commented 2 years ago

Hi, I see that someone has reproduced your experimental results, but I haven't been able to reproduce them. I would like to ask if the version of python, cuda, pytorch has a big impact on the results? Because I use a 3090 graphics card, the version of cuda and pytorch cannot be consistent with the author. cuda==11.0 torch==1.7 detectron2==0.3 Only the number of GPUs is modified in the code, and the others are not modified. The base model provided by the author is used in the reproduction process, and the experimental results of the reproduction are quite different from the author. And it fluctuates a lot.

fenghuo-cao commented 2 years ago

I have tested G_FSOD's split_1 1_shot seed_0 many times. Your paper only mentions the experimental result of taking the average of ten experiments. Your experimental result is 57.03, while mine is between 50-55. What is the cause of this problem? of? Have you encountered similar problems during your experiments?

RuoyuChen10 commented 2 years ago

Hi, I see that someone has reproduced your experimental results, but I haven't been able to reproduce them. I would like to ask if the version of python, cuda, pytorch has a big impact on the results? Because I use a 3090 graphics card, the version of cuda and pytorch cannot be consistent with the author. cuda==11.0 torch==1.7 detectron2==0.3 Only the number of GPUs is modified in the code, and the others are not modified. The base model provided by the author is used in the reproduction process, and the experimental results of the reproduction are quite different from the author. And it fluctuates a lot.

I haven't tested the code yet, but according to my experience, based on the framework of Detectron2 implementation, there is a big difference between the implementation results of 1 3090 graphics card and 2 2080Ti graphics cards. I guess it is because of Detectron2 optimized multi-card training.

2 3090 may work, you can try graphics 2 cards.

fenghuo-cao commented 2 years ago

@RuoyuChen10 Ok, I see what you mean, but I don't have a machine with two 3090 graphics cards. I'll try running on 2 2080Ti graphics cards in a while. thank you.

CrapbagMo commented 2 years ago

Hi, I came up with the same issue. Have you reproduced the results yet in 2 cards setting?

IceHowe commented 2 years ago

@RuoyuChen10 Ok, I see what you mean, but I don't have a machine with two 3090 graphics cards. I'll try running on 2 2080Ti graphics cards in a while. thank you.

Can you tell me how much time it takes to train a epoch on COCO using 2 2080TI?

kangkang189 commented 1 year ago

I run your code under 2080 with batchsize=2 per gpu,i got the lowwer Ap ,how can i get the ap like your paper?

STSTERANDMOMO commented 1 year ago

Hi, I see that someone has reproduced your experimental results, but I haven't been able to reproduce them. I would like to ask if the version of python, cuda, pytorch has a big impact on the results? Because I use a 3090 graphics card, the version of cuda and pytorch cannot be consistent with the author. cuda==11.0 torch==1.7 detectron2==0.3 Only the number of GPUs is modified in the code, and the others are not modified. The base model provided by the author is used in the reproduction process, and the experimental results of the reproduction are quite different from the author. And it fluctuates a lot.

I met a problem。rm: cannot remove 'checkpoints/voc/defrcn/defrcn_fsod_r101_novel1/tfa-like/5shot_seed9/model_final.pth': No such file or directory。can you help me？

ruee-z commented 1 year ago

Hi, I see that someone has reproduced your experimental results, but I haven't been able to reproduce them. I would like to ask if the version of python, cuda, pytorch has a big impact on the results? Because I use a 3090 graphics card, the version of cuda and pytorch cannot be consistent with the author. cuda==11.0 torch==1.7 detectron2==0.3 Only the number of GPUs is modified in the code, and the others are not modified. The base model provided by the author is used in the reproduction process, and the experimental results of the reproduction are quite different from the author. And it fluctuates a lot.

May I ask if you later reproduced the results of the paper and how did you do it? I also encountered the same problem now, I ran on two 2080 with bs=4

xsh8757398 commented 1 year ago

I have the same problem. There is roughly a 3-7 difference between my reproduced results and those in Table 6 with AP, AP50, AP75, bAP, and nAP. Does the machine used have much to do with the results? I am using two 1080s with a batch size of 4.