Sparse representation of likelihood tensors (`A` and `B` arrays)

[conorheins]

• allow users to specify an optional list of hidden state factor indices when passing in A and B arrays, that specify which hidden state factors correspond to the lagging dimensions of these tensors. The purpose of this feature is to speed up computation speed and limit memory consumption. It leverages the (often-satisfied) assumption in generative models that not all observation modalities depend on all hidden state factors. This means we don't have to include many of the lagging dimensions in the likelihood tensors that correspond to hidden state factors that the observation modality in question doesn't depend on. The same goes for the B arrays, which are already accordingly sparse due to the baked-in mean-field approximation of the posterior (statistical independence of hidden state factors in the prior).

  • I imagine these structures should be two list of lists (one for A and one for B), structured as follows:
    • A_factor_list, where A_factor_list[m] contains a list of the hidden state factor indices that correspond to the factors that observation modality m is conditionally dependent on.
    • B_factor_list where B_factor_list[f] is the list of hidden state factor indices that are needed to predict the dynamics of factor f. For now we will assume control factors are still fully conditionally independent, such that one B[f] sub-array only depends on the control state factor with index f, even though it may depend on other hidden state factors.

• completing the above will also require building in the option of having additional dependencies in each factor-specific B array, on the other hidden state factors.

[conorheins]

This is currently implemented in this branch: https://github.com/infer-actively/pymdp/tree/sparse_likelihoods_111

But I will leave the issue open until it's merged into master

[conorheins]

Now merged into master via PR #132