Prior information as `scipy.stats`

[TTitscher]

Defining priors is currently only supported for VariationalBayesProblem and restricted to uncorrelated normal distributions for the parameters and gamma distributions for the noise models. This works OKish for automatically converting a VariationalBayesProblem into a sampling problem. Users that do not care about Chappell's algorithm, are likely puzzled by this.

I propose:

• Provide prior distributions as scipy.stats distributions (norm, gamma, ...) directly to the InferenceProblem.
• Provide a SolverInterface that is implemented by i.e. VBSolver or PyMC3Solver. Those are more like example implementations with a fixed, limited set of options to control to underlying solvers. More experienced users will likely write/interface their own way of handing the InferenceProblem.

[joergfunger]

That is a good idea. Maybe we could directly connect this prior definitions to our latent variables definition, such that we define a latent variable with an optional prior (in case someone wants to do deterministic optimization), and then link it to local variables in the model error/forward model class(es).

[TTitscher]

Especially regarding my second point with the example implementations, I'd say that the definition on what is latent is problem specific and the (somehow crafted) question how it is latent (initial deterministic value, logpdf, custom MVN) is then solver specific.

[joergfunger]

Alright, that is a true. So we'll try to separate the inference problem into on the one hand the handling of latent variables (with all the global/local variable distinction) and model error, and on the other hand specific interfaces for solvers that use that.

[TTitscher]

I think I had a conceptual mistake in my last comment. My plan would be that our inference problem does include prior information, say mathematically with the information N(42, ...) or Gamma(21, 42) or Crystalball(...). The individual solvers then transform this information into tool-specific implementations of these distributions.

And just to not redefine every (common) distribution out there, I suggest to store the information via scipy.stats and the implementation is then provided in a:

• VBSolver -- transforms it to custum MVN / Gamma
• TaralliSolver -- can use the scipy.stats objects right away
• PyroSolver -- transforms to pyro.distributions

[joergfunger]

I would agree doing this, but we should allow users to just use the latent/model_error feature (so that they provide the priors themselves), e.g. via methods that can be overwritten. Thus, special structures do not have to be part of the standard interface. Important is the consideration of vector parameters. Here, the question would be, if we consider each parameter separately (and provide a single scalar distribution assuming all entries in the vector are independent and idenically distributed, or if we directly provide a joint distribution (e.g. MVN) with the special case of only have diagonal entries in the covariance.

[TTitscher]

Good point. IMO the vector valued parameters are a very special case related to FEMU. Maybe we continue the discussion on that in #41