New additions to pymdp that support updating dirichlet parameters (pD) over prior over initial states. This is the first step towards implementing "structure learning" in pymdp, which will consist into doing Bayesian model reduction using a combination of the posterior beliefs about the Dirichlet parameters of the generative model likelihoods (e.g. pA or pB) and posterior beliefs about the Dirichlet parameters of the generative model priors (for now, pD).
Additions include:
new function for updating pD in learning.py module
New method of Agent() class, called update_D(), that is compatible with the various sorts of posterior inference schemes available to theAgent(). I.e. it should work with both MMP-style inference (where posterior beliefs about both future and past hidden states, under all policies, are entertained) as well as "vanilla" inference (instantaneous posterior beliefs about only the current timestep, unconditioned on policies).
New additions to
pymdpthat support updating dirichlet parameters (pD) over prior over initial states. This is the first step towards implementing "structure learning" inpymdp, which will consist into doing Bayesian model reduction using a combination of the posterior beliefs about the Dirichlet parameters of the generative model likelihoods (e.g.pAorpB) and posterior beliefs about the Dirichlet parameters of the generative model priors (for now,pD).Additions include:
pDinlearning.pymoduleAgent()class, calledupdate_D(), that is compatible with the various sorts of posterior inference schemes available to theAgent(). I.e. it should work with both MMP-style inference (where posterior beliefs about both future and past hidden states, under all policies, are entertained) as well as "vanilla" inference (instantaneous posterior beliefs about only the current timestep, unconditioned on policies).