oseiskar / simdkalman

Python Kalman filters vectorized as Single Instruction, Multiple Data
https://simdkalman.readthedocs.io/
MIT License
176 stars 36 forks source link

Question: How to use this with a dataframe? #21

Closed BillGatesIII closed 3 years ago

BillGatesIII commented 3 years ago

Amazing stuff and I'm curious if I can use it with pandas dataframe. For example I have this dataframe

d = {'A': [8, 2, 4, 5, 6, 4, 3, 5, 5, 3]}
df = pd.DataFrame(data=d)

print(df)

which looks like this

   A
0  8
1  2
2  4
3  5
4  6
5  4
6  3
7  5
8  5
9  3

How can I add a row with the Kalman prediction for row 10?

I came this far

kf = simdkalman.KalmanFilter(
    state_transition = [[1,1],[0,1]],        # matrix A
    process_noise = np.diag([0.1, 0.01]),    # Q
    observation_model = np.array([[1,0]]),   # H
    observation_noise = 1.0)

pr = kf.predict(df.squeeze(), 1)

to get one predicted value.

But adding it to the dataframe gives an error.

df.append(pr)

TypeError: cannot concatenate object of type '<class 'simdkalman.kalmanfilter.KalmanFilter.Result'>'; only Series and DataFrame objs are valid

I understand why I get the error but I don't know how to 'extract' the predicted value from object.

oseiskar commented 3 years ago

The return type of predict is documented here.

In you are interested in the prediction only (not the uncertainty or internal state of the filter), you can append the .observations.mean to your Dataframe, for example, like this

df.append(pd.DataFrame(
    data={ 'A': pr.observations.mean },
    index=[10]))

But please note that for good results, the matrices in the Kalman filter (A,Q,H & R) should be well thought out and tuned for your specific problem. The matrices in example.py are not intended to work as a general purpose prediction algorithm.