feature request: state-space models

[dvanamstel]

Add state-space models, in discrete form:

x(k+1) = A * x(k) + B * u(k)
y(k) = C * x(k) + D * u(k)

in where: x(k) - internal state vector at timestamp k u(k) - input vector at timestamp k y(k) - output at timestamp k

Initial implementation would be with a given size of state vector x (e.g. you know the dimension of the underlying system). Second iteration could also estimate the length of this vector x, but that's prob not doable in a single day.

Must admit: I haven't seen many use-cases that would be best solved using a state-space model and thus wonder how useful this can be. Also, I haven't seen many use-cases in general.

[koaning]

A few observations.

1. This assumes that the dataset X, y needs to be sorted and needs to represent a single timeseries? Is this the intended behavior?
2. What is reasonable for k=0?
3. How many steps ahead might be reasonable to predict?
4. Can you come up with a use-case?

[dvanamstel]

1. Indeed, X and y have to be sorted time series.

2. Not sure if I understand the question correctly? For k=0 you insert the first element in your time-series, e.g. the "initial condition".

3. In theory; infinite. In practice: depends on how well the fit is.

Compared to a linear model (Note that if A, B & C are 0 the formulation reduces to a "standard" linear model y = D * u) it should perform better as it learns time-domain dynamics. I wouldn't be surprised if you can achieve the same result by adding delayed features...but that is manual work ;).

4.

• Marketing efforts to sales predictions
• Production line (or any system that has "B can start after A is finished" types of rules) disturbance analysis.

[koaning]

in a way, it sounds as if you are going to implement an RNN.

[koaning]

i might be open to something like this being a part of scikit-lego via a dependency. but i kind of want to be careful with how many dependencies we add to the project.

thoughts, @MBrouns ?

[dvanamstel]

I wasn't planning on implementing a RNN. That would be equivalent to using a neural network to implement a linear model: possible but a bit overkill. :)

[koaning]

something tells me that this will be really hard to implement if you consider the train/test aspect of it. ill close this issue for now (but feel free to challenge me on this)