search

facebookresearch / adaptive_teacher

This repo provides the source code for "Cross-Domain Adaptive Teacher for Object Detection".
Other
180 stars 35 forks source link

AP NaN #27

Closed darkhan-s closed 2 years ago

darkhan-s commented 2 years ago

Hello,

I formed a new target dataset in Pascal VOC format and as I understand, the target dataset should be unlabeled so I did not add .xml files to the Annotations folder of the target dataset. But how does the evaluation for the unlabeled images work in the teacher model if there are no ground truth boxes?

Specifically, at every EVAL_PERIOD iteration this line returns NaN: https://github.com/facebookresearch/adaptive_teacher/blob/d57d20640ae314a42c43dd82b1c1e26e90fa4b95/adapteacher/evaluation/pascal_voc_evaluation.py#L305

What should be done instead? Thanks!

yujheli commented 2 years ago

Hi, I think you need to provide the annotations for the testing dataset for evaluation. For the unlabelled "training" dataset, you may need to give dummy annotation files to avoid errors.

darkhan-s commented 2 years ago

@yujheli thanks, that works. But then does it mean that the method is rather semi-supervised than fully-unsupervised? Or in the end the performance would be the same without any labels, despite the difficulties in evaluating?

yujheli commented 2 years ago

@darkhan-s Due to the code pipeline of detectron2, the label files (regarding ground truths or dummy ones) are supposed to be provided. Techinically the model does not use any labels from unlabelled datasets so the performance would be the same if I understand your question.

darkhan-s commented 2 years ago

Yes, that answers my question. Thank you!