question about coco_unseen_ids

dyabel / detpro

Apache License 2.0

174 stars 28 forks source link

question about coco_unseen_ids #7

Closed Feobi1999 closed 2 years ago

Feobi1999 commented 2 years ago

Hi, I want to know what's the meaning of the coco_unseen_ids_train, coco_unseen_ids_test in the file class_name.py. It seems that it is different from the popular (17+48) zero-shot setting. eg. The coco_unseen_ids_test list contains 15 items. Thanks!

dyabel commented 2 years ago

Hi, I want to know what's the meaning of the coco_unseen_ids_train, coco_unseen_ids_test in the file class_name.py. It seems that it is different from the popular (17+48) zero-shot setting. eg. The coco_unseen_ids_test list contains 15 items. Thanks!

Hi, it means the removed classes during test phase. We need to remove 17+15 classes during training and 15 classes during testing. So the 15 classes are removed all the time. Here, 'unseen' means 'removed', sorry for causing the confusion.

Feobi1999 commented 2 years ago

Thank you for your quick reply! Looks like you also conduct experiments on COCO zero-shot setting? Could you share the config file for this? It will help me a lot.

Feobi1999 commented 2 years ago

@dyabel Hi, I read your code and really think it is a clear implementation. Now I want to ask another question. I think the implementation of StandardRoIHeadTEXTPrompt is related to the VILD-text part. I run experiments with this ROI head on coco zero-shot setting but get a bad result. (the all classes mAP50=44.9 but novel classes mAP50=0.384) Other configs are consistent with this file 'configs/transfer/faster_rcnn_r50_fpn_sample1e-3_mstrain_voc_pretrain.py' The AP in the figure below means AP50.

Have you tried the only vild-text experiments during your research? I don't know if the results are normal now or if I haven't tuned the hyperparameters properly.

yestinl commented 2 years ago

@dyabel Hi, I read your code and really think it is a clear implementation. Now I want to ask another question. I think the implementation of StandardRoIHeadTEXTPrompt is related to the VILD-text part. I run experiments with this ROI head on coco zero-shot setting but get a bad result. (the all classes mAP50=44.9 but novel classes mAP50=0.384) Other configs are consistent with this file 'configs/transfer/faster_rcnn_r50_fpn_sample1e-3_mstrain_voc_pretrain.py' The AP in the figure below means AP50. Have you tried the only vild-text experiments during your research? I don't know if the results are normal now or if I haven't tuned the hyperparameters properly.

@Feobi1999 Hi，did you fix this issue?

Feobi1999 commented 2 years ago

@dyabel Not yet, did you also get such a strange result?