search

ComputationalPsychiatry / pyhgf

PyHGF: A neural network library for predictive coding
https://computationalpsychiatry.github.io/pyhgf/
GNU General Public License v3.0
59 stars 19 forks source link

Predictive Coding vs. HGF ? #255

Open cgoemaere opened 6 days ago

cgoemaere commented 6 days ago

I found this repo while looking for a Predictive Coding library, inspired by this paper. However, the focus here seems to be more on Generalized Hierarchical Gaussian Filters (HGF).

Could you clarify how HGFs relate to Predictive Coding? Are HGFs a specific instance of Predictive Coding, or do they represent a broader framework?

PTWaade commented 2 days ago

This probably warrants a deeper both technical and conceptual investigation, and I hesitate to claim much expertise in predictive coding in general. That said, I would say that the generalized HGF can be seen as one instance of a predictive coding network, with some differences between the traditional types of predictive coding networks. One advantage of the HGF over at least some types of predictive coding networks (primarily from talking to Rafal Bogacz; I may have understood plenty of things) is that the HGF uses one-step update equations where other approaches use gradient descent on free energy. This leads to the HGF having a similar but slightly different microcircuitry. More details: https://arxiv.org/abs/2305.10937 Another is that HGF'es can include temporal dynamics i.e. volatilities and drifts - and that it can include volatility learning for dynamically adjusting the precision and learning rates in the Bayesian update. But it doesn't have to. So in this regard, classical Gaussian predictive coding networks might be (or at least be similar to) a special type of the generalized HGF.

Finally, pyhgf as a package is made to be able to in principle implement other kinds of predictive coding networks than the HGF. So it should in principle be possible to implement other types of predictive coding networks too.

In general, it would be exciting to compare these different variations of predictive coding properly. Let us know if I'm mistaken in any of the above; if there is any interest in comparing the HGF to other predictive coding networks; if there is anything else we can help with; and if there is anything interesting worth sharing with us :)

Thanks for writing!