search

AdaptiveMotorControlLab / CEBRA

Learnable latent embeddings for joint behavioral and neural analysis - Official implementation of CEBRA
https://cebra.ai
Other
908 stars 77 forks source link

Sampling scheme for continuous auxiliary variables clarification #29

Closed mariakesa closed 1 year ago

mariakesa commented 1 year ago

Hello,

Thank you so much for your work! I am definitely having fun with CEBRA:-)

I just wanted to verify whether I understood the sampling scheme for shaping embeddings using an auxiliary variable. I read the supplement to the paper and looked at the code. Do I understand correctly that if I specify a continuous auxiliary variable, the model will create a distribution of delta in values between neighboring time points and then use this distribution to find the closest sample to the reference sample with this delta as a positive sample (the model still only learns from the main data ie the data that I would apply the time only algorithm to, except now I get the positive samples using the auxiliary variable)? Are the negative samples still sampled uniformly as in time only learning?

Thanks again!

MMathisLab commented 1 year ago

thanks @mariakesa for the question; just a first small note to say we have Discussions open for questions: https://github.com/AdaptiveMotorControlLab/CEBRA/discussions , so in the future that would be an ideal place to post questions and leave issues to bug reports

mariakesa commented 1 year ago

I apologize, I didn't notice the discussions forum.

Thank you so much! I'm glad I understood correctly!

stes commented 1 year ago

To this point,

Do I understand correctly that if I specify a continuous auxiliary variable, the model will create a distribution of delta in values between neighboring time points and then use this distribution to find the closest sample to the reference sample with this delta as a positive sample

That is correct. What you describe is implemented here:

https://github.com/AdaptiveMotorControlLab/CEBRA/blob/6409239dac8563e0c5838391b244ae34eefbe547/cebra/distributions/continuous.py#L231-L242

However, there are other possibilities, like sampling around the reference samples, as done in this case:

https://github.com/AdaptiveMotorControlLab/CEBRA/blob/6409239dac8563e0c5838391b244ae34eefbe547/cebra/distributions/continuous.py#L271-L285

(the model still only learns from the main data ie the data that I would apply the time only algorithm to, except now I get the positive samples using the auxiliary variable)?

Yes, it simply "re-indexes" the dataset you provide to fit()