Vector valued parameters

[TTitscher]

A relevant use case for vector valued parameters would be a FE forward model, where all displacements or stresses are latent variables it is just easier to define the whole field with possibly millions of entries.

The library somehow supports vector valued parameters, but there is no example. Also, actually going to 1e6 parameters in the current implementation, causes issues regarding:

• dense Jacobian with 1e6 x 1e6 entries
• dense precision/covariance matrixes with 1e6 x 1e6 entries
• matrix multiplication i.e. Jacobian.T @ Jacobian that is tricky even if the Jacobian is sparse
• any matrix inversions

So do we want to support that in a general fashion in our InferenceProblem or would that rather be a special case only relevant for specifically tailored VB algorithm? I vote for the latter.

[joergfunger]

I would say those are two separate questions. The general interface should allow this, the restriction you mention all relate to our VB implementation (with optional options to work on all the issues you mentioned above). Thus, if our VB is just a single inference solver, the vector parameter approach would generally be very helpful, e.g. I had a problem with model bias being modelled by a correlated Gaussian process, and all the amplitudes at the training points where random variables. For sure, this works only for small problems with the currently implemented inference solvers, but a parameter vector would be very convenient.

[joergfunger]

Another application is a random field, so the model parameters are the individual parameters (spatially variable, but correlated).

[TTitscher]

Further discussed here https://github.com/BAMresearch/probeye/issues/2