facebookresearch / adaptive_teacher

This repo provides the source code for "Cross-Domain Adaptive Teacher for Object Detection".
Other
183 stars 37 forks source link

why ap ap50 ap75 = 0.000? #20

Closed oopsboy closed 2 years ago

oopsboy commented 2 years ago

Train

oopsboy commented 2 years ago

222

wha-cloud commented 2 years ago

222

I have the same problem as you, did you solve it? and my total loss became nan

yujheli commented 2 years ago

@wha-cloud Could you check if the learning rate is too large? Or if you use too small batch size? See: https://github.com/facebookresearch/adaptive_teacher/issues/9

wha-cloud commented 2 years ago

@wha-cloud Could you check if the learning rate is too large? Or if you use too small batch size? See: #9

Thanks! When I set the batch size to 8, up to now 2000 iterations are no problem. If it is set to 4, there are problems with several hundred iterations. Now I have a new question, it seems to be evaluated twice when training, why is the result of the second evaluation 0?

image
oopsboy commented 2 years ago

@wha-cloud I wonder if you have solved it?

yujheli commented 2 years ago

@wha-cloud @oopsboy Does the number in tensorboard make sense? I always looked at the number in tensorboard and rarely see the numbers printed out.

oopsboy commented 2 years ago

@wha云@oopsboy张量板中的数字有意义吗?我总是在张量板中查看数字,很少看到打印出来的数字。

@yujheli Yes, when I test with the trained model, its map is also 0, so this problem exists

helq2612 commented 2 years ago

@wha-cloud I guess you are in the source only pre-training stage, in which the student model does not have learned weights yet (https://github.com/facebookresearch/adaptive_teacher/blob/cba3c59cadfc9f1a3a676a82bf63d76579ab552b/adapteacher/engine/trainer.py#L506).

oopsboy commented 2 years ago

@wha-cloud I guess you are in the source only pre-training stage, in which the student model does not have learned weights yet (

https://github.com/facebookresearch/adaptive_teacher/blob/cba3c59cadfc9f1a3a676a82bf63d76579ab552b/adapteacher/engine/trainer.py#L506

).

@helq2612 Thank you for your answer. How can I modify the number of times of this paramete?

helq2612 commented 2 years ago

@oopsboy I think you can modify it in the config files, e.g.: https://github.com/facebookresearch/adaptive_teacher/blob/cba3c59cadfc9f1a3a676a82bf63d76579ab552b/configs/faster_rcnn_R101_cross_clipart.yaml#L43 Or pass it as args to train_net.py

yujheli commented 2 years ago

Ok, the model will be evaluated two times: If you are in the stage of burn-in, you will get 0 AP for teacher. bbox (not uses):

image

bbox_student:

image
oopsboy commented 2 years ago

ok,3q!