Inaccuracy in the README?

[markisus]

The README states "The problem is that one cannot write an observation equation for 2D feature observations in image space for a localization kalman filter." Maybe I'm interpreting it wrong, but it's actually very common for observation functions to take the state to a lower dimension. For example a filter for estimating position and velocity may only take positions as measurements.

[haraschax]

I understand your confusion, I will think about rewording it.

It is true that you can write an observation function for a lower dimensionality than the state. In this case the problem is that the 2D feature observation depends on both the position of the camera and the position of the feature. For the observation to be correct both those would have to be in the state, since both are unknowns. This is a possible solution, but not very practical as the state would need to be large enough to contain the positions of all features being tracked and the features need to be constantly replaced when new features come into view.

MSCKF provides an alternative that does not require the feature positions to be in the state. https://www-users.cs.umn.edu/~stergios/papers/ICRA07-MSCKF.pdf

[markisus]

Oh interesting paper, thanks for the reference. Instead of incorporating the measurements into the state, the feature observations are projected onto a subspace where errors in the feature positions will lead to no changes in the residual.