search

rlabbe / Kalman-and-Bayesian-Filters-in-Python

Kalman Filter book using Jupyter Notebook. Focuses on building intuition and experience, not formal proofs. Includes Kalman filters,extended Kalman filters, unscented Kalman filters, particle filters, and more. All exercises include solutions.
Other
16.74k stars 4.2k forks source link

Ch. 8 Evaluating Filter Order - "Residual" Plots? #315

Closed PLeVasseur closed 7 months ago

PLeVasseur commented 5 years ago

Wanted to start off with a big thank you for the helpful resource on Kalman Filters.

In the Evaluating Filter Order section, there's discussion around how plotting the residual against the corresponding standard deviation element for the state we're looking at is useful to check filter performance. When I look at the code that's written there I see that what's being computed and plotted is not the residual, but the estimation error.

If we think about it, that's the only way that we're able to make a "residual" plot of the velocity, since we cannot find the residual of the velocity. We cannot find the residual of the velocity because there is no measurement of velocity to subtract a prior from. Does this make sense? Am I missing something?

gparmeggiani commented 4 years ago

I completely agree. In fact I opened (and now closed since it is a duplicate of this) issue #318

I think that instead of "residual" he should have used "error". errors can be compared with the variances in P, while residuals can't since residuals are in the measurement space, while P is in the state space.

The residuals (measurement space) could be compared to variances in the the "system uncertainty" covariance matrix (S = HPH' + R, Where P is the P of the prior; in other words, S is the denominator of the Kalman Gain).

More formally, these two comparisons can be performed with the computation of NEES (as seen in the book) and NIS Similarly to the NEES, the NIS is computed as y' S^-1 y The key difference between the two is that you can compute the NIS at run time, while you can't do the same with the NEES because, obviously, you don't know the ground truth (see my issue #280)

Also note that the NIS is used as part of the equation to compute the likelihood function

rlabbe commented 4 years ago

This is so old, perhaps you have moved on.

However, this section is specifically about testing using a simulated system, in which case we have ground truth. We thus know the 'actual' position and velocity, since we are simulating it.

As I wrote

I must mention that this is only true for simulated systems.

I agree with the comments in #318, this paragraph can be expanded to distinguish between simulated and real data.

gparmeggiani commented 4 years ago

Hi Roger, I think I still have somewhere the changes I made to the paragraph. I could make a PR and take it from there

rlabbe commented 4 years ago

That would be helpful, thanks!

PLeVasseur commented 7 months ago

Closing as resolved :slightly_smiling_face: