Can symforce be used in a sliding window-based non-linear optimization system?

[wxliu1]

How to use symforce to generate marginalization prior factors? In ther words, how to use symforce to add marginalization factors to the optimizer?

[aaron-skydio]

The tools are all there, but we don't currently have them bundled together into a single function or set of functions to call for this. The process would with the current API would be something like:

1. Collect the sym::Factors touching the variables you want to marginalize
2. Use a sym::Linearizer to linearize those factors (you could do this manually also, but using the Linearizer is probably easier)
3. Eliminate the marginalized variables in the resulting linearization. The sym::SchurComplementSolver doesn't quite expose what you'd need to do this, so for now you'd do this just with e.g. linear algebra ops from Eigen
4. Create a new sym::Factor with the resulting linearization after elimination, and use that in the next optimization

[empty-spacebar]

The tools are all there, but we don't currently have them bundled together into a single function or set of functions to call for this. The process would with the current API would be something like:

1. Collect the sym::Factors touching the variables you want to marginalize
2. Use a sym::Linearizer to linearize those factors (you could do this manually also, but using the Linearizer is probably easier)
3. Eliminate the marginalized variables in the resulting linearization. The sym::SchurComplementSolver doesn't quite expose what you'd need to do this, so for now you'd do this just with e.g. linear algebra ops from Eigen
4. Create a new sym::Factor with the resulting linearization after elimination, and use that in the next optimization

Symforce doesn't have an official implementation of marginalization, is it because that the manipulation of matrix which is no longer sparse after being marginalized, can't get noticeable accelerated with symforce?