search

mboudiaf / pytorch-meta-dataset

A non-official 100% PyTorch implementation of META-DATASET benchmark for few-shot classification
59 stars 9 forks source link

Unexpected behavior from min_examples_in_class #6

Closed jfb54 closed 3 years ago

jfb54 commented 3 years ago

The official meta-dataset does not have the parameter min_examples_in_class exposed (as far as I know). I set it to 1 as I wanted any class to have at least one example and I get the error message when I turn on use_bilevel_hierarchy for Omniglot (which is standard for Meta-Dataset).

"use_bilevel_hierarchy" is incompatible with "min_examples_in_class"

I don't understand why this restriction is required.

mboudiaf commented 3 years ago

So this condition actually comes from the original code at https://github.com/google-research/meta-dataset/blob/ec57aeb8356d9c1c8dfb0cc6edfe23a4eb2c4614/meta_dataset/data/sampling.py#L314 .

I am not really sure why this condition has to be satisfied to be honest, and found no further explanation in the source code. If I had to guess though, I would say that all classes in Omniglot have exactly 20 samples, and so if the user sets this argument to < 20, nothing will happen, and if >= 20, all classes will be skipped. So this incompatibility may have just be put in place as a precaution.

jfb54 commented 3 years ago

Got it. Thanks for the clarification.