DeLightCMU / CASD

85 stars 21 forks source link

Question about consistency loss!!! #18

Closed suilin0432 closed 3 years ago

suilin0432 commented 3 years ago

Hi~ Does consistency_conf_loss really work? Firstly, I run your code as you stated in README.md and found the fact that consistency_conf_loss is really small. So I guess that consistency_conf_loss may have no contribution to the performance. Then I tried remove the consistency_conf_loss and run your "baseline" again. I got nearly the same performance 53+ even when the training process had not been finished. I wonder that the really important part of your codebase is your tricks which are not mentioned in your paper or the consistency_conf_loss.

Cynthiacoding commented 3 years ago

Hello! I'm also a learner about CSAD, but I have not successfully executed it limited the GPU source. Could you please share your log? Thanks! By the way, I sent you an email. Looking forward to your reply!

suilin0432 commented 3 years ago

Hello! I'm also a learner about CSAD, but I have not successfully executed it limited the GPU source. Could you please share your log? Thanks! By the way, I sent you an email. Looking forward to your reply!

You should use GPU with large memory, such as V100, 3090 or M40. I think I have replied you, maybe you should check your spam.

Justinhzy commented 3 years ago

Hi, Thanks for your question. Our method is built upon MELM paper. It contains some methods introduced by MELM that we may not mention in our paper. You can find those details in MELM repo (https://github.com/vasgaowei/pytorch_MELM).

About the consistency loss, did you change any hyper-parameters or random seed? The result should like this VOC07 metric? Yes AP for aeroplane = 0.7051 AP for bicycle = 0.7011 AP for bird = 0.5704 AP for boat = 0.4575 AP for bottle = 0.2945 AP for bus = 0.7453 AP for car = 0.7277 AP for cat = 0.7144 AP for chair = 0.2531 AP for cow = 0.6763 AP for diningtable = 0.4930 AP for dog = 0.6474 AP for horse = 0.6577 AP for motorbike = 0.7268 AP for person = 0.2372 AP for pottedplant = 0.2586 AP for sheep = 0.5631 AP for sofa = 0.6077 AP for train = 0.6536 AP for tvmonitor = 0.6650 Mean AP = 0.5678

Results:
0.705
0.701
0.570
0.457
0.294
0.745
0.728
0.714
0.253
0.676
0.493
0.647
0.658
0.727
0.237
0.259
0.563
0.608
0.654
0.665
0.568
suilin0432 commented 3 years ago

Fine~. But I didn't change any thing, maybe this codebase is not fully consistent with your own code which could achieve the result stated in your paper. By the way it's also strange that adding the reg branch could only boost the performace less than 1 AP. But anyway, the result reported in your parper is really good, could you please give me the checkpoint of COCO which could reach 13.9.