Can't find the parameter to specify the "task-irrelevant" and "-relevant variables"

[YukunQu]

Is there an existing issue for this?

• [X] I have searched the existing issues

Bug description

Hello, I have some task-irrelevant variables to control in the embedding process. This is a crucial aspect of your paper. However, I am unable to find a parameter to specify it. I have thoroughly searched all the tutorials of CEBRA. It appears that the PyTorch API, Distributions, may be able to handle it. Nevertheless, I have not yet come across a clear example or parameter to do so.

Perhaps I haven't found it. Would you mind helping me? I sincerely seek your assistance.

Operating System

Ubuntu 20.04.2 LTS

CEBRA version

'0.4.0'

Device type

gpu

Steps To Reproduce

No response

Relevant log output

No response

Anything else?

No response

Code of Conduct

• [X] I agree to follow this project's Code of Conduct

[YukunQu]

In your paper: "mixed single-session sampling could be extended additionally to incorporate a dimension to which the algorithm should become invariant; this would add an additional step of uniform sampling with regard to this desired discrete variable (for example, via ancestral sampling)."

Is 'mixed single-session sampling' capable of achieving this (control task-irrelevant variables)? Moreover, can it be utilized in multi-session training conditions?

If there is a simple example for this, I will be very appreciated.

[MMathisLab]

You simply decide what auxiliary variables to include in the positive samples; for example with multi-session training we don't use animal ID as a variable, as we want to become invariant to it. The data sampling is therefore critically tied to the model optimization.

[YukunQu]

You simply decide what auxiliary variables to include in the positive samples; for example with multi-session training we don't use animal ID as a variable, as we want to become invariant to it. The data sampling is therefore critically tied to the model optimization.

Thank you for your response! Perhaps I didn't explain my question clearly. When it comes to task-relevant variables, I believe we can simply input them into the model for fitting, as shown in the code below. But...

multi_cebra_model = CEBRA(model_architecture='offset10-model', batch_size=512, learning_rate=3e-4, temperature=1, output_dimension=3, max_iterations=max_iterations, distance='cosine', conditional='time_delta', device='cuda_if_available', verbose=True, time_offsets=10) multi_cebra_model.fit(datas, labels)

In this example, the labels contain the variables that are task-relevant, such as position and direction. However, how can I include the labels for task-irrelevant variables in training? I understand that CEBRA can manage the session/animal ID for this purpose. Nevertheless, I have another confounding variable for each animal. How can I provide the label to ensure that the model is invariant to it? which parameter is right? If you can answer me, it will be very helpful!

[MMathisLab]

Aside from changing negative sampling, there is no explicit "ignore this" input variable; in our paper we should how not-specifically putting in variables (like animal ID) is the appropriate proxy for this.

You can edit the source code of course to explicitly sample the negative distribution.

[YukunQu]

Aside from changing negative sampling, there is no explicit "ignore this" input variable; in our paper we should how not-specifically putting in variables (like animal ID) is the appropriate proxy for this.

You can edit the source code of course to explicitly sample the negative distribution.

I understand. Thank you! Perhaps I can make the necessary changes and submit a 'pull request' for this feature

To clarify, in accordance with your paper： “Optionally, with a categorical context variable , negative sampling can be performed to approximate a uniform distribution of samples over this context variable. If this is performed for both negative and positive samples, the resulting embedding will become invariant with respect to the variable kt. ”,

To ensure that the model remains invariant to task-irrelevant variables (such as reward), I should perform uniform sampling for both positive and negative samples. Am I understanding this correctly?

[MMathisLab]

Hi @YukunQu this is no PR needed, it's fully in the code :). You simply design your positive and negative distributions as delineated in the docs, hope that helps!