Revisit notation?

[GordStephen]

(This issue may be a bit ephemeral given #54, or it may spill over to there as well.) As our state space models become more complex to accommodate exogenous inputs, time-dependent system matrices, and constrained parameter estimation, it may be worth revisiting the notation used in the source code. Unfortunately notation for state space modelling tends to be all over the map - things vary widely both across and within stats and (to a lesser degree) engineering.

As of right now, our system equations are given as:

x[t] = F[t-1] * x[t-1] + B[t-1] * u[t-1] + v[t-1], v ~ N(0, V[t-1])
y[t] = G[t]   * x[t]   + D[t]   * u[t]   + w[t], w ~ N(0, W[t])

Time-independent EM parameter estimation for partially-deterministic systems will require at least two new system matrices, and supporting parameter estimation for time-dependent system matrices will require several more matrices as well. As the system grows more complicated it becomes increasingly useful for notation to be consistent with other references - meanwhile, as the functionality and usefulness of the package increases, familiarity for potential users or contributors becomes equally (and maybe more) important.

Taking all that into consideration, I'd propose the following, attempting to harmonize source code with both (reasonably) standardized control engineering notation (A, B, C, D) and what I'm seeing most often in the relevant stats literature (v, w, Q, R, G, H):

x[t] = A[t-1] * x[t-1] + B[t-1] * u[t-1] + v[t-1]
y[t] = C[t]   * x[t]   + D[t]   * u[t]   + w[t]

A[t] = A1[t] * A2 * A3[t]
B[t] = B1[t] * B2
C[t] = C1[t] * C2 * C3[t]
D[t] = D1[t] * D2
v[t] = G[t] * q[t], q[t] ~ N(0, Q)
w[t] = H[t] * r[t], r[t] ~ N(0, R)

A2, B2, C2, D2, Q, and R are time-independent matrices that can be parametrized and estimated. A1, A3, B1, C1, C3, D1, G, and H are time-dependent matrices that must be explicitly defined, and of course that remains the case for y and u as well.

Are there any objections to moving to this notation when EM estimation is implemented? Alternatively, feel free to suggest alternate conventions?

[GordStephen]

Also, with all the system matrices, instead of having StateSpaceModel constructors with 6, 8, 10 arguments, switching some arguments to kwargs might provide both a simpler API and a cleaner implementation (fewer explicit methods) - in that case the notation would be directly exposed to the user, so all the more reason to choose wisely.

[GordStephen]

This was adopted in #57.