search

Vibashan / irg-sfda

Official Pytorch codebase for Instance Relation Graph Guided Source-Free Domain Adaptive Object Detection [CVPR 2023]
https://viudomain.github.io/irg-sfda-web/
63 stars 7 forks source link

AP is always 0 #8

Closed AnnaK-M closed 1 year ago

AnnaK-M commented 1 year ago

Hi Mr. Vibashan,

I would like to reproduce your paper but I only get AP values of 0 when I test with your uploaded model with this code:

python tools/plain_test_net.py --eval-only --config-file configs/sfda/foggy_baseline.yaml --model-dir model/model_teacher_10.pth

I get the following output: [08/10 15:36:07 fvcore.common.checkpoint]: [Checkpointer] Loading from model/model_teacher_10.pth ... WARNING [08/10 15:36:07 fvcore.common.checkpoint]: Some model parameters or buffers are not found in the checkpoint: GraphCN.gc1.{bias, weight} GraphCN.gc2.{bias, weight} GraphCN.gc3.{bias, weight} GraphCN.graph.wk.{bias, weight} GraphCN.graph.wq.{bias, weight} Graph_conloss.head_1.0.{bias, weight} Graph_conloss.head_1.2.{bias, weight} Graph_conloss.head_2.0.{bias, weight} Graph_conloss.head_2.2.{bias, weight} [08/10 15:36:07 detectron2]: Trained model has been sucessfully loaded ... [08/10 15:37:25 detectron2]: Evaluation results for cityscape_2007_test_t in csv format: [08/10 15:37:25 d2.evaluation.testing]: copypaste: Task: bbox [08/10 15:37:25 d2.evaluation.testing]: copypaste: AP,AP50,AP75 [08/10 15:37:25 d2.evaluation.testing]: copypaste: 0.0000,0.0002,0.0000

I also get an AP of 0 after the training with foggy cityscapes with the base source model. I have not altered the code. Do you have any idea what the problem could be?

Thank you for your work and kind regards, Anna

Vibashan commented 1 year ago

Hi @AnnaK-M ,

Can you try to visualize the results using the pre-trained weights?

AnnaK-M commented 1 year ago

Hi @Vibashan,

thanks for the quick response, when I plot the images with predicitons they look something like this:

result_1

It's strange that the predictions have high scores but are really bad. I always get the warning from above, WARNING [08/10 15:36:07 fvcore.common.checkpoint]: Some model parameters or buffers are not found in the checkpoint:, when loading the model. Is that normal?

Do you have any suggestions what the problem might be?

Thanks!

Vibashan commented 1 year ago

Hi @AnnaK-M,

It looks like there might be a problem with the dataloader or post-processing script. To debug, you could:

  1. Visualize GT using the Val dataloader.
  2. If that works, then most probably the issue is with the post-processing script.

Regarding missing model parameters, it's normal for certain components not to be in the checkpoint. This is expected for as RCNN does not have student network components such as the IRG network.

Let me know if you need more help!

AnnaK-M commented 1 year ago

The issue was that width and height were interchanged and therefore the scaling in the post processing was wrong.

Thanks for the help!