search

mazpie / contrastive-aif

[NeurIPS 2021] Contrastive learning formulation of the active inference framework, for matching visual goal states.
https://contrastive-aif.github.io/
MIT License
8 stars 1 forks source link

Posterior computation #2

Open sai-prasanna opened 1 month ago

sai-prasanna commented 1 month ago

In your posterior, you use the stochastic state of the prior. But in RSSM they only use the deterministic state, and observation embedding. Since the prior's stochastic state is just a function of the deterministic state, it won't have extra information to condition upon. And using the stochastic state sample might hurt computing the posterior because of the sampling noise.

I am checking in case there is some other deeper reason to use it.

https://github.com/mazpie/contrastive-aif/blob/980e386400dccc56f8879e3aa866d98a869364f9/world_model.py#L129

mazpie commented 1 month ago

Hi @sai-prasanna, you're right that there is a subtle difference with the original RSSM.

However, I would not expect any major differences as the information to condition upon is contained in the deterministic state, as you pointed out.

The stochastic state might either be helpful (it is a more noisy estimate of the state) or be ignored by the network, if it doesn't contain any useful information (e.g. if you just concatenate random noise to the inputs of a network, the network quickly learns to ignore it)

I hope this answers your question!