search

Sha-Lab / FEAT

The code repository for "Few-Shot Learning via Embedding Adaptation with Set-to-Set Functions"
MIT License
418 stars 84 forks source link

the ues of temperature #23

Closed liubingg closed 4 years ago

liubingg commented 4 years ago

Thanks for your code! But i have a question, what's the meaning of the use of the temperature? Thank you!

Han-Jia commented 4 years ago

Hi, liubingg,

The temperature is important when using the pre-trained embedding due to a scale mismatch. The temperature could be thought as a kind of calibration.

TADAM also provides a way to think about the temperature.

liubingg commented 4 years ago

Thanks, i got it. But how to calculate the value of the temperature?

Han-Jia commented 4 years ago

Empirically, I find ProtoNet with pretrain needs large temperature (e.g., 32, 64, or even 128), while MatchNet needs a small temperature.

Based on my experience, the temperature will reduce the loss value during the initial training stages.