OID-V6 dataset preprocessing

[Leterax]

Amazing paper, especially so soon after the last one, great work! I had one question regarding the oidv6 dataset: did you do any preprocessing to filter out bad classes etc? How many of the 9 million images did you end up using? Also, are there any large differeneces between train.py (COCO) and what you used to train on the oidv6 dataset?

Thanks in advance.

[ebenbaruch]

Thanks for the kind words :)

We used the 9,605 trainable classes as provided in the official OpenImages dataset. We used only the human annotations, ending up with almost 6 million images. The training code for COCO and OpenImages datasets is exactly the same. The main difference was the hyper-parameters used (as the focusing parameters).

[Leterax]

Thanks for the quick reply, can i ask what kind of process you used for loading the data, did you just store each image with its one-hot-encoded labels? Also, did you do any comparisons to other backbones? (maybe efficientnetV2)

[ebenbaruch]

We used the meta-files provided in the OpenImages official webpage (https://storage.googleapis.com/openimages/web/factsfigures.html). Specifically, aligning the image_ids to their positive and negative annotations. We didn't encode the labels to 1/0 vectors as it is not recommended (memory-wise). I suggest storing the label indexes and encoding them on the fly during the training.

Regarding comparing to other backbones, as you can find in table 2 of the paper, we provided results for multiple backbones (including Resnet or OFA (efficient NAS architecture) based models).

[Leterax]

Ah, i had some trouble finding literature about OFA, but thanks! Yeah I agree on the label encoding, that's also how i have been doing it. I will probably come back with more questions at some point, but that's it for now ;)

[HITESHLPATEL]

Hey, @ebenbaruch Thanks for sharing your great work. I have a question regarding training on OpenImages. Did you use Negative annotations during training for eg each image comes with a label and confidence of 0 and 1? So did only take positive annotations i.e Label with confidence 1?

[ebenbaruch]

Yes, we used the negative annotations as well. Each sample is annotated with a few positive labels and a few negative labels. The rest of the labels are unknown.

[HITESHLPATEL]

Thanks, @ebenbaruch . Currently how I train is I use only positive labels I,e the one with Confidence 1, and create a one-hot encoding for the image i.e [0,0,1,0....n9605] and train the model. What changes do I need to do if I also want to incorporate negative labels during training? Thank you

[ebenbaruch]

The approach you mentioned refers to the "negative" mode. If you like to consider also the annotated negative labels, you can use the baseline training mode: --partial_loss_mode=ignore, or our method: --partial_loss_mode=selective. You may want to look at the PartialSelectiveLoss for more details: (https://github.com/Alibaba-MIIL/PartialLabelingCSL/blob/534f3ce1997607ceaa2db8eb5d06986f490ed88b/src/loss_functions/partial_asymmetric_loss.py#L71).