Closed mikediamore closed 6 years ago
Hi,
Great to hear that you have found the library useful! Unfortunately, what you are attempting is not possible. This is not due to this library but how Kalman filters and smoothers work. Let me explain...
If you have a sequence of T values, [y1, y2, ..., yT], the output of the Kalman filter at element t, 1 ≤ t ≤ T depends on the past values only, that is, y1, y2, ..., yt - 1, yt. The state of the Kalman filter can be efficiently updated online using only the previous state (mt-1, Pt-1) and the new value yt (using kf.update
in this library).
Conversely, the value of the Kalman smoother (the output of kf.smooth
) at sequence element t depends on all the past and the future values: y1, y2, ..., yt, ..., yT. It cannot be computed online becase every value in the sequence affects every output value of the Kalman smoother.
Here's an example (code follows the image) where the Kalman filter (dashed line) and smoother (solid line) for the same model have been applied to the full data with 150 elements (black) and the first 100 elements (blue).
A few things to notice:
import numpy as np
import simdkalman
import pandas as pd
import matplotlib.pyplot as plt
np.random.seed(0)
data = np.random.normal(size=150)
kf = simdkalman.KalmanFilter(
state_transition = [[1,1],[0,1]], # matrix A
process_noise = np.diag([0.05, 0.002]), # Q
observation_model = np.array([[1,0]]), # H
observation_noise = 20.0)
kf_results = kf.smooth(data)
t = range(data.size)
def kf_smooth(y):
return kf.smooth(y).observations.mean
def kf_filter(y):
return kf.compute(y, 0, filtered=True).filtered.observations.mean
plt.plot(t, data, 'kx', alpha=0.2, label='data')
plt.plot(t[:100], data[:-50], 'bx', alpha=0.4, label='first 100')
plt.plot(t, kf_smooth(data), 'k-', label='smoothed full')
plt.plot(t, kf_filter(data), 'k--', label='filtered full')
plt.plot(t[:100], kf_smooth(data[:100]), 'b-', label='smoothed first 100')
plt.plot(t[:100], kf_filter(data[:100]), 'b--', label='filtered first 100')
plt.legend()
plt.show()
I hope this clarifies the problem.
BR, Otto
Hello,
This library has been quite helpful! I was wondering how to properly call the update and smooth_current functions to update the smoother in an online fashion.
For example, if I have the following to smooth 10000 normal random variables
And I wanted to get the smoothed value for the 10,001st term would I do the following?
I'm finding that the above doesn't do a very good job of recreating the smoothing on the whole series especially if I continue this process over longer time periods. For example:
Here are the results from the last 500 points of the kf.smooth(data) call
And here are the results from using the following for loop, adapting #Snippet 2 above
Is it possible to recreate the output of kf.smooth() using kf.update() and kf.smooth_current()?
Here's the full code example to make it easier to run