Some question about evaluate

[zml110120]

❓ Some question about evaluate

I'm using my own data set (only 14 categories), but pred_classes in the red box below is greater than 14. What's the problem? How can I modify it? I have been registered our dataset in detectron2/ data/datasets/cdod.py .

[XuYunqiu]

Could you please provide more details of your datasets?

1. Which datasets are used as the source and target datasets respectively?
2. How many categories are there in the two datasets?
3. Which pre-trained model is used for evaluation?

[zml110120]

@XuYunqiu Dear author,

1. cp_train cp_val are my own datasets. I use cp_train as train target, voc_2012_trainval as train source, and cp_val as test. As shown in the first picture.

2. cp_train cp_val datasets both have 14 categories.

3. The pre-train model uses the already generated model (trained by cp_train voc_2012_trainval). But during pre-train phase, i had not change num_classes in yaml file. As shown in the second picture. I consulted you for advice in other issue https://github.com/XuYunqiu/H2FA_R-CNN/issues/8#issuecomment-1365742212 , but there is still some problems.

[XuYunqiu]

The categories in both source and target datasets should be the same, in terms of total number and class index.

The source dataset voc_2012_trainval is the standard VOC dataset with 20 classes, while the target datasets cp_train and cp_val have 14 classes. During training, the detector is learning to predict 20 classes. Thus, you may get the out-of-index issue.

You can solve this issue by:

1. Besides registering cp_train and cp_val, you should also register a custom VOC dataset with the same class number and class index with cp_train and cp_val.
2. Setting NUM_CLASSES to 14, in both training and inference stages.

[zml110120]

@XuYunqiu Dear author, I will try your advice! I still have some questions:

1. Order of pred_classes wherether same as CLASS_NAMES_14? As shown in first picture.

2. The origin code can save model per 5000 iters. How can i change it to specified iters? Such as save model per 2000 iters?

[XuYunqiu]

1. The order of them is the same. But the class index is from 0 to 13, if there are 14 classes. Another thing is that our H²FA R-CNN currently only supports CDWSOD which assumes the source and target domains have the SAME label space. I'm not sure if it still works well, when the target domain contains some novel classes.

2. You can set SOLVER.CHECKPOINT_PERIOD to 2000 in the config file.

[zml110120]

Hi, In your model evaluation phase, the batch size looks like equal 1. How can i change the evaluation phase batch size?

[XuYunqiu]

It's not supported as https://detectron2.readthedocs.io/modules/data.html#detectron2.data.build_detection_test_loader says, so writing some custom code will be needed.

Perhaps, you can do it by hacking the code here https://github.com/XuYunqiu/H2FA_R-CNN/blob/499c1d90833c475fd9f288bb6bb90829989def64/detectron2/data/build.py#L393-L396 Just change the batch size in the BatchSampler.

[XuYunqiu]

I'm closing this issue, but let me know if you have further questions.