search

ServiceNow / embedding-propagation

Codebase for Embedding Propagation: Smoother Manifold for Few-Shot Classification. This is a ServiceNow Research project that was started at Element AI.
Apache License 2.0
208 stars 21 forks source link

Effect of Batch Size #25

Closed Dhul-Husni closed 2 years ago

Dhul-Husni commented 2 years ago

Hi, awesome project.

For embedding propagation what is your intuition on how the batch size affects the intra-cluster homogeneity and inter-cluster separation? This is not with regards to training but with regards to EP being a function applied to the results of a feature extractor. I can imagine smaller batch sizes lead to more local homogeneity and larger batch sizes consider global homogeneity, is my intuition correct? Is there a systematic way to find a good batch size?

prlz77 commented 2 years ago

Hi thanks!

We applied embedding propagation to each episode independently without considering batches, thus the batch size has no influence on the results of EP. However, the shot number does have an effect! This is not studied in depth in our paper. And we found the best alpha by cross-validation.

Hope this clarifies your question,

Pau

Dhul-Husni commented 2 years ago

Thanks Pau, this does.