low accuracy on unseen classes

[chiran7]

Dear author,

I trained and got good results for Coco. Setting split = unseen, test accuracy for both seen and unseen classes can be obtained with gzsd. However, in Pascal VOC config, for the test set, should we train separately to get test accuracy on seen and unseen classes? Because, while I trained making split = seen, it only shows seen class accuracy during testing and also only for unseen class (in both zsd and gzsd), while trying for split = unseen. Moreover, while trying on custom data, mAP for seen class is very hi in the range of 60-70. During training GAN(step 4), validation accuracy is higher (around 25.00), but test accuracy is very low (around 5.00). Can you please suggest a solution for higher accuracy on seen class and lower accuracy on the unseen class for the test (although validation accuracy is a little higher for the unseen class)?

Thank you for your time and consideration.

[sandipan211]

Yes, we followed the settings used in previous works and tested separately for seen and unseen.

Regarding custom data, the generated visual features from GAN are used to train a classifier whose weights are copied to the FasterRCNN to empower it with unseen class detection. Have you done that step?

We also recommend you go through these instructions we uploaded recently in our README.md for handling custom data. If you have missed any of the steps mentioned there or replaced them with some other step of your own, the result and behavior of the model might be affected.