search

BayesWatch / deep-kernel-transfer

Official pytorch implementation of the paper "Bayesian Meta-Learning for the Few-Shot Setting via Deep Kernels" (NeurIPS 2020)
https://arxiv.org/abs/1910.05199
197 stars 29 forks source link

Loaded DKT model performs poorly. #23

Open sambatra opened 7 months ago

sambatra commented 7 months ago

Hi there,

I was wondering why the saved and loaded model performs poorly compared to the original model. I used the exact same test data in both cases and got R2_score = 0.97, -0.001 for the original and loaded model respectively.

Best

mpatacchiola commented 7 months ago

Hi @sambatra . Can you please explain better what you mean by "original" and "loaded" model? Is the original model the result in the paper and loaded the one you obtain by training yourself?

A common issue could be due to the version of Pytorch and GPytorch used in your experiments. Try using the same setup we used and the same seeds, the results should be exactly the same.