search

nv-research-israel / causal_comp

This repository hosts the dataset and source code for "A causal view of compositional zero-shot recognition". Yuval Atzmon, Felix Kreuk, Uri Shalit, Gal Chechik (Spotlight)
Other
27 stars 2 forks source link

4 of the splits miss at least one attribute in the training data #3

Open FrankRuis opened 3 years ago

FrankRuis commented 3 years ago

I noticed that the 3:7, 5:5, and 7:3 splits all have one color that is never seen in the training data and the 6:4 split training data misses 3 colors.

I was wondering if this is intentional, as I thought this violates the CZSL assumptions that all individual attributes and objects are seen and e.g. makes it impossible to reach over ~40% accuracy on the unseen data of the first two 6:4 splits.

More specifically, the proportions of (samples with unseen attribute / total samples) for the unseen samples of the test sets of the following seeds are: 900/2400 for seed 3000 and 3002, seed 3001 is good 900/3600 for seed 5000 and 5002, seed 5001 is good 2700/4500 for seed 6000 and 6001, 1800/4500 for seed 6002 900/5100 for seed 7000, 2700/5100 for seed 7001 and 7002

yuvalatzmon commented 3 years ago

Hi Frank,

That's a good point. We didn't want to add an additional source of bias, so the splits we made were completely random. Missing some colors in the training set is an artifact of this design choice.

Best, Yuval