Open empty-spacebar opened 1 month ago
It's hard to say much here without knowing a lot more about what you're doing (what do you mean when you say graph-based VSLAM, what factors are included, what initialization, etc).
It would also be useful to know what optimization parameters are being used with Ceres, or if Ceres is selecting any of those automatically, which it's choosing (which optimizer, which linear solver, inner iterations, etc).
It looks like you've tried the DYNAMIC lambda parameter, which is one thing that can improve convergence in more nonlinear problems like bundle adjustment.
One other thing that would be useful to know is whether the SymForce optimizer converges eventually, and how many iterations it takes, if you increase the iteration limit.
It's hard to say much here without knowing a lot more about what you're doing (what do you mean when you say graph-based VSLAM, what factors are included, what initialization, etc).
The graph-based VSLAM framwork is VINS-MONO, feature-based frontend, factor-graph backend optimized with Gauss–Newton originally, and it optimizes camera poses, velocity, IMU bias, inverse-depth of landmark points.
It would also be useful to know what optimization parameters are being used with Ceres, or if Ceres is selecting any of those automatically, which it's choosing (which optimizer, which linear solver, inner iterations, etc).
Here is the Ceres parameter that works well in VINS:
options.max_num_iterations = 8
options.linear_solver_type = ceres::DENSE_SCHUR;
options.trust_region_strategy_type = ceres::DOGLEG
Symforce doesn't have DOGLEG, so I replace ceres::DOGLEG with LM in symforce like this:
using DenseOptimizer =
sym::Optimizer<Scalar, sym::LevenbergMarquardtSolver<Scalar, sym::DenseCholeskySolver<Scalar>>>;
I also tried the default sparse decomposition:
sym::Optimizer<double> optimizer
None of them runs well.
Maybe the contrast between Ceres and Symforce comes from the performance difference between LM and dogleg?
So here comes the question----Will Symforce support more iterative optimization algorithm like dogleg in the future?
It looks like you've tried the DYNAMIC lambda parameter, which is one thing that can improve convergence in more nonlinear problems like bundle adjustment.
DYNAMIC lambda parameter doesn't work----with it VINS runs even worse. Thus static lambda is used as the 2nd image above.
One other thing that would be useful to know is whether the SymForce optimizer converges eventually, and how many iterations it takes, if you increase the iteration limit.
As for whether the SymForce optimizer converges eventually, it's showed in the fourth image above----seldom converges, usually disconverges and ends up with hitting iteration limit(50 times iterations).
Again I'm not sure how much we're going to be able to debug this over GitHub issues, so this may be a dead end, but a couple more thoughts:
Here is the Ceres parameter that works well in VINS
Is Ceres using inner iterations? It might help to have the Ceres solver output, e.g. ceres::Solver::Summary::FullReport()
. It might be useful to run Ceres with LM and no inner iterations, which would be closer to the SymForce optimizer.
DYNAMIC lambda parameter doesn't work----with it VINS runs even worse. Thus static lambda is used as the 2nd image above.
I'm not too shocked by this, but I'll note that DYNAMIC is basically what Ceres does by default in their LM implementation, while STATIC is not
As for whether the SymForce optimizer converges eventually, it's showed in the fourth image above----seldom converges, usually disconverges and ends up with hitting iteration limit(50 times iterations).
What I was asking is, if you set the SymForce iteration limit much higher, what happens? It's much more useful to know what SymForce would do with an unlimited number of iterations (does it converge in 60? 100? 1000? or get stuck?) (does optimizing to convergence give the same solution as Ceres?) than knowing that it often takes more than 50.
It is often also useful to see the full verbose optimization trace (by setting the verbose
parameter to true
), to know whether the optimizer is making good progress, and if not why not. Separately, it can be useful to visualize the values at intermediate steps of the optimization (you can save the Values for each iteration in the OptimizationStats by turning on debug_stats
, and you seem to have some visualization tools set up already that you could probably use for this)
Is Ceres using inner iterations? It might help to have the Ceres solver output, e.g. `ceres::Solver::Summary::FullReport()
I didn't set the option 'use_inner_iterations', and since its default value is false, I guess inner iterations is OFF in my cases. I pack all my test log up, I hope the file names are clear enough to know where they came from. Also I took the screenshots of the trajectories of every test. symforce_test_20240726.zip
What I was asking is, if you set the SymForce iteration limit much higher, what happens? It's much more useful to know what SymForce would do with an unlimited number of iterations (does it converge in 60? 100? 1000? or get stuck?) (does optimizing to convergence give the same solution as Ceres?) than knowing that it often takes more than 50.
Log of this test is also being record into the file above. It's interesting that when Ceres runs well, it doesn't converge either(mostly in 4 times iterations, end because iteration time out). The reason why symforce runs worse is because its residual decreases VERY SLOWLY, sometimes takes 50 times iteration to get a good result(see sym_LM_dense_cholesky.png), sometimes just crush.
And more interesting is, Ceres doesn't run well everytimes(see Ceres_LM_dense_cholesky1~3, which means exactly the same parameter but different results)
Here are some other discoveries: 1, Ceres runs awfully with LM, no matter dense schur or dense cholesky is used; 2, Ceres with LM and sparse cholesky, reports error like 'dense matrix decomposition fail' but still get right trajectory; 3, symforce with LM and sparse cholesky runs bad, with dense cholesky and 50 times iterations runs good. 4, When set lambda zero, the program report:'Internal Error: Damped hessian factorization failed.'
That's actually pretty helpful, thanks
Optimization traces like this are exactly those that should be improved by dynamic lambda:
[2024-07-26 14:11:26.634] [info] LM<sym::Optimize> [iter 0] lambda: 1.000e+00, error prev/linear/new: 2.010e+02/2.009e+02/2.010e+02, rel reduction: 2.45473e-04, gain ratio: 5.14165e-01
[2024-07-26 14:11:26.636] [info] LM<sym::Optimize> [iter 1] lambda: 1.000e-01, error prev/linear/new: 2.010e+02/2.008e+02/2.011e+02, rel reduction: -4.78170e-04, gain ratio: -6.30568e-01
[2024-07-26 14:11:26.639] [info] LM<sym::Optimize> [iter 2] lambda: 1.000e+00, error prev/linear/new: 2.010e+02/2.009e+02/2.009e+02, rel reduction: 1.02077e-04, gain ratio: 3.61970e-01
[2024-07-26 14:11:26.641] [info] LM<sym::Optimize> [iter 3] lambda: 1.000e-01, error prev/linear/new: 2.009e+02/2.008e+02/2.011e+02, rel reduction: -6.08975e-04, gain ratio: -1.06610e+00
[2024-07-26 14:11:26.643] [info] LM<sym::Optimize> [iter 4] lambda: 1.000e+00, error prev/linear/new: 2.009e+02/2.009e+02/2.009e+02, rel reduction: 4.45069e-05, gain ratio: 2.27182e-01
[2024-07-26 14:11:26.645] [info] LM<sym::Optimize> [iter 5] lambda: 1.000e-01, error prev/linear/new: 2.009e+02/2.008e+02/2.011e+02, rel reduction: -6.62745e-04, gain ratio: -1.44154e+00
[2024-07-26 14:11:26.647] [info] LM<sym::Optimize> [iter 6] lambda: 1.000e+00, error prev/linear/new: 2.009e+02/2.009e+02/2.009e+02, rel reduction: 1.60077e-05, gain ratio: 1.08418e-01
[2024-07-26 14:11:26.650] [info] LM<sym::Optimize> [iter 7] lambda: 1.000e-01, error prev/linear/new: 2.009e+02/2.008e+02/2.011e+02, rel reduction: -6.79496e-04, gain ratio: -1.75608e+00
[2024-07-26 14:11:26.652] [info] LM<sym::Optimize> [iter 8] lambda: 1.000e+00, error prev/linear/new: 2.009e+02/2.009e+02/2.009e+02, rel reduction: 1.89187e-06, gain ratio: 1.61017e-02
[2024-07-26 14:11:26.654] [info] LM<sym::Optimize> [iter 9] lambda: 1.000e-01, error prev/linear/new: 2.009e+02/2.009e+02/2.011e+02, rel reduction: -6.77233e-04, gain ratio: -2.01352e+00
[2024-07-26 14:11:26.656] [info] LM<sym::Optimize> [iter 10] lambda: 1.000e+00, error prev/linear/new: 2.009e+02/2.009e+02/2.009e+02, rel reduction: -4.29621e-06, gain ratio: -4.41172e-02
[2024-07-26 14:11:26.659] [info] LM<sym::Optimize> [iter 11] lambda: 1.000e+01, error prev/linear/new: 2.009e+02/2.009e+02/2.009e+02, rel reduction: 1.68324e-05, gain ratio: 1.27198e+00
[2024-07-26 14:11:26.661] [info] LM<sym::Optimize> [iter 12] lambda: 1.000e+00, error prev/linear/new: 2.009e+02/2.009e+02/2.009e+02, rel reduction: -4.27360e-06, gain ratio: -4.47309e-02
[2024-07-26 14:11:26.664] [info] LM<sym::Optimize> [iter 13] lambda: 1.000e+01, error prev/linear/new: 2.009e+02/2.009e+02/2.009e+02, rel reduction: 1.65771e-05, gain ratio: 1.28204e+00
[2024-07-26 14:11:26.666] [info] LM<sym::Optimize> [iter 14] lambda: 1.000e+00, error prev/linear/new: 2.009e+02/2.009e+02/2.009e+02, rel reduction: -4.21685e-06, gain ratio: -4.49676e-02
[2024-07-26 14:11:26.668] [info] LM<sym::Optimize> [iter 15] lambda: 1.000e+01, error prev/linear/new: 2.009e+02/2.009e+02/2.009e+02, rel reduction: 1.63717e-05, gain ratio: 1.29433e+00
[2024-07-26 14:11:26.671] [info] LM<sym::Optimize> [iter 16] lambda: 1.000e+00, error prev/linear/new: 2.009e+02/2.009e+02/2.009e+02, rel reduction: -4.13596e-06, gain ratio: -4.49176e-02
[2024-07-26 14:11:26.673] [info] LM<sym::Optimize> [iter 17] lambda: 1.000e+01, error prev/linear/new: 2.009e+02/2.009e+02/2.009e+02, rel reduction: 1.62052e-05, gain ratio: 1.30846e+00
[2024-07-26 14:11:26.676] [info] LM<sym::Optimize> [iter 18] lambda: 1.000e+00, error prev/linear/new: 2.009e+02/2.009e+02/2.009e+02, rel reduction: -4.03752e-06, gain ratio: -4.46405e-02
[2024-07-26 14:11:26.678] [info] LM<sym::Optimize> [iter 19] lambda: 1.000e+01, error prev/linear/new: 2.009e+02/2.009e+02/2.009e+02, rel reduction: 1.60693e-05, gain ratio: 1.32411e+00
[2024-07-26 14:11:26.681] [info] LM<sym::Optimize> [iter 20] lambda: 1.000e+00, error prev/linear/new: 2.009e+02/2.009e+02/2.009e+02, rel reduction: -3.92577e-06, gain ratio: -4.41744e-02
[2024-07-26 14:11:26.683] [info] LM<sym::Optimize> [iter 21] lambda: 1.000e+01, error prev/linear/new: 2.009e+02/2.009e+02/2.009e+02, rel reduction: 1.59577e-05, gain ratio: 1.34101e+00
[2024-07-26 14:11:26.685] [info] LM<sym::Optimize> [iter 22] lambda: 1.000e+00, error prev/linear/new: 2.009e+02/2.009e+02/2.009e+02, rel reduction: -3.80335e-06, gain ratio: -4.35422e-02
[2024-07-26 14:11:26.687] [info] LM<sym::Optimize> [iter 23] lambda: 1.000e+01, error prev/linear/new: 2.009e+02/2.009e+02/2.009e+02, rel reduction: 1.58654e-05, gain ratio: 1.35893e+00
[2024-07-26 14:11:26.689] [info] LM<sym::Optimize> [iter 24] lambda: 1.000e+00, error prev/linear/new: 2.009e+02/2.009e+02/2.009e+02, rel reduction: -3.67189e-06, gain ratio: -4.27562e-02
[2024-07-26 14:11:26.691] [info] LM<sym::Optimize> [iter 25] lambda: 1.000e+01, error prev/linear/new: 2.009e+02/2.009e+02/2.009e+02, rel reduction: 1.57885e-05, gain ratio: 1.37768e+00
[2024-07-26 14:11:26.693] [info] LM<sym::Optimize> [iter 26] lambda: 1.000e+00, error prev/linear/new: 2.009e+02/2.009e+02/2.009e+02, rel reduction: -3.53228e-06, gain ratio: -4.18221e-02
[2024-07-26 14:11:26.696] [info] LM<sym::Optimize> [iter 27] lambda: 1.000e+01, error prev/linear/new: 2.009e+02/2.009e+02/2.009e+02, rel reduction: 1.57240e-05, gain ratio: 1.39712e+00
[2024-07-26 14:11:26.698] [info] LM<sym::Optimize> [iter 28] lambda: 1.000e+00, error prev/linear/new: 2.009e+02/2.009e+02/2.009e+02, rel reduction: -3.38502e-06, gain ratio: -4.07408e-02
[2024-07-26 14:11:26.700] [info] LM<sym::Optimize> [iter 29] lambda: 1.000e+01, error prev/linear/new: 2.009e+02/2.009e+02/2.009e+02, rel reduction: 1.56696e-05, gain ratio: 1.41712e+00
[2024-07-26 14:11:26.702] [info] LM<sym::Optimize> [iter 30] lambda: 1.000e+00, error prev/linear/new: 2.009e+02/2.009e+02/2.009e+02, rel reduction: -3.23032e-06, gain ratio: -3.95103e-02
[2024-07-26 14:11:26.705] [info] LM<sym::Optimize> [iter 31] lambda: 1.000e+01, error prev/linear/new: 2.009e+02/2.009e+02/2.009e+02, rel reduction: 1.56235e-05, gain ratio: 1.43758e+00
[2024-07-26 14:11:26.707] [info] LM<sym::Optimize> [iter 32] lambda: 1.000e+00, error prev/linear/new: 2.009e+02/2.009e+02/2.009e+02, rel reduction: -3.06825e-06, gain ratio: -3.81271e-02
[2024-07-26 14:11:26.709] [info] LM<sym::Optimize> [iter 33] lambda: 1.000e+01, error prev/linear/new: 2.009e+02/2.009e+02/2.009e+02, rel reduction: 1.55842e-05, gain ratio: 1.45842e+00
[2024-07-26 14:11:26.712] [info] LM<sym::Optimize> [iter 34] lambda: 1.000e+00, error prev/linear/new: 2.009e+02/2.009e+02/2.009e+02, rel reduction: -2.89880e-06, gain ratio: -3.65868e-02
[2024-07-26 14:11:26.714] [info] LM<sym::Optimize> [iter 35] lambda: 1.000e+01, error prev/linear/new: 2.009e+02/2.009e+02/2.009e+02, rel reduction: 1.55506e-05, gain ratio: 1.47957e+00
[2024-07-26 14:11:26.717] [info] LM<sym::Optimize> [iter 36] lambda: 1.000e+00, error prev/linear/new: 2.009e+02/2.009e+02/2.009e+02, rel reduction: -2.72195e-06, gain ratio: -3.48846e-02
[2024-07-26 14:11:26.721] [info] LM<sym::Optimize> [iter 37] lambda: 1.000e+01, error prev/linear/new: 2.009e+02/2.009e+02/2.009e+02, rel reduction: 1.55218e-05, gain ratio: 1.50097e+00
[2024-07-26 14:11:26.724] [info] LM<sym::Optimize> [iter 38] lambda: 1.000e+00, error prev/linear/new: 2.009e+02/2.009e+02/2.009e+02, rel reduction: -2.53767e-06, gain ratio: -3.30160e-02
[2024-07-26 14:11:26.726] [info] LM<sym::Optimize> [iter 39] lambda: 1.000e+01, error prev/linear/new: 2.009e+02/2.009e+02/2.009e+02, rel reduction: 1.54970e-05, gain ratio: 1.52259e+00
[2024-07-26 14:11:26.729] [info] LM<sym::Optimize> [iter 40] lambda: 1.000e+00, error prev/linear/new: 2.009e+02/2.009e+02/2.009e+02, rel reduction: -2.34597e-06, gain ratio: -3.09769e-02
[2024-07-26 14:11:26.732] [info] LM<sym::Optimize> [iter 41] lambda: 1.000e+01, error prev/linear/new: 2.009e+02/2.009e+02/2.009e+02, rel reduction: 1.54756e-05, gain ratio: 1.54438e+00
[2024-07-26 14:11:26.734] [info] LM<sym::Optimize> [iter 42] lambda: 1.000e+00, error prev/linear/new: 2.009e+02/2.009e+02/2.009e+02, rel reduction: -2.14689e-06, gain ratio: -2.87635e-02
[2024-07-26 14:11:26.737] [info] LM<sym::Optimize> [iter 43] lambda: 1.000e+01, error prev/linear/new: 2.009e+02/2.009e+02/2.009e+02, rel reduction: 1.54573e-05, gain ratio: 1.56632e+00
[2024-07-26 14:11:26.739] [info] LM<sym::Optimize> [iter 44] lambda: 1.000e+00, error prev/linear/new: 2.009e+02/2.009e+02/2.009e+02, rel reduction: -1.94050e-06, gain ratio: -2.63729e-02
[2024-07-26 14:11:26.742] [info] LM<sym::Optimize> [iter 45] lambda: 1.000e+01, error prev/linear/new: 2.009e+02/2.009e+02/2.009e+02, rel reduction: 1.54416e-05, gain ratio: 1.58837e+00
[2024-07-26 14:11:26.745] [info] LM<sym::Optimize> [iter 46] lambda: 1.000e+00, error prev/linear/new: 2.009e+02/2.009e+02/2.009e+02, rel reduction: -1.72693e-06, gain ratio: -2.38027e-02
[2024-07-26 14:11:26.747] [info] LM<sym::Optimize> [iter 47] lambda: 1.000e+01, error prev/linear/new: 2.009e+02/2.009e+02/2.009e+02, rel reduction: 1.54282e-05, gain ratio: 1.61053e+00
[2024-07-26 14:11:26.750] [info] LM<sym::Optimize> [iter 48] lambda: 1.000e+00, error prev/linear/new: 2.009e+02/2.009e+02/2.009e+02, rel reduction: -1.50632e-06, gain ratio: -2.10511e-02
[2024-07-26 14:11:26.752] [info] LM<sym::Optimize> [iter 49] lambda: 1.000e+01, error prev/linear/new: 2.009e+02/2.009e+02/2.009e+02, rel reduction: 1.54169e-05, gain ratio: 1.63278e+00
[2024-07-26 14:11:26.752] [warning] LM<sym::Optimize> Optimization finished with status: HIT_ITERATION_LIMIT
Traces like this one are a little weird, but I think may be resolved by using a smaller min lambda - the gain ratio is consistently high (>>1), which is a bit weird but arguably good (the optimizer is making much more progress than it expects based on the current linearization). This maybe suggests the optimizer is making good progress even though the problem is highly nonlinear. Anyway, I'd try decreasing the min lambda.
[2024-07-26 14:11:26.768] [info] LM<sym::Optimize> [iter 0] lambda: 1.000e+00, error prev/linear/new: 2.310e+02/2.309e+02/2.309e+02, rel reduction: 1.74157e-04, gain ratio: 1.19171e+00
[2024-07-26 14:11:26.770] [info] LM<sym::Optimize> [iter 1] lambda: 1.000e-01, error prev/linear/new: 2.309e+02/2.309e+02/2.308e+02, rel reduction: 4.50963e-04, gain ratio: 1.44795e+00
[2024-07-26 14:11:26.773] [info] LM<sym::Optimize> [iter 2] lambda: 1.000e-02, error prev/linear/new: 2.308e+02/2.307e+02/2.308e+02, rel reduction: 2.02472e-04, gain ratio: 4.42505e-01
[2024-07-26 14:11:26.776] [info] LM<sym::Optimize> [iter 3] lambda: 1.000e-03, error prev/linear/new: 2.308e+02/2.307e+02/2.307e+02, rel reduction: 2.31655e-04, gain ratio: 7.88559e-01
[2024-07-26 14:11:26.778] [info] LM<sym::Optimize> [iter 4] lambda: 1.000e-04, error prev/linear/new: 2.307e+02/2.307e+02/2.306e+02, rel reduction: 5.22702e-04, gain ratio: 8.66064e+00
[2024-07-26 14:11:26.782] [info] LM<sym::Optimize> [iter 5] lambda: 1.000e-05, error prev/linear/new: 2.306e+02/2.306e+02/2.305e+02, rel reduction: 6.61930e-04, gain ratio: 2.43387e+01
[2024-07-26 14:11:26.785] [info] LM<sym::Optimize> [iter 6] lambda: 1.000e-06, error prev/linear/new: 2.305e+02/2.305e+02/2.303e+02, rel reduction: 6.44922e-04, gain ratio: 3.96905e+01
[2024-07-26 14:11:26.788] [info] LM<sym::Optimize> [iter 7] lambda: 1.000e-07, error prev/linear/new: 2.303e+02/2.303e+02/2.302e+02, rel reduction: 6.61305e-04, gain ratio: 1.66814e+01
[2024-07-26 14:11:26.790] [info] LM<sym::Optimize> [iter 8] lambda: 1.000e-08, error prev/linear/new: 2.302e+02/2.301e+02/2.300e+02, rel reduction: 8.40862e-04, gain ratio: 5.89807e+00
[2024-07-26 14:11:26.792] [info] LM<sym::Optimize> [iter 9] lambda: 1.000e-08, error prev/linear/new: 2.300e+02/2.300e+02/2.298e+02, rel reduction: 8.02504e-04, gain ratio: 1.42988e+01
[2024-07-26 14:11:26.795] [info] LM<sym::Optimize> [iter 10] lambda: 1.000e-08, error prev/linear/new: 2.298e+02/2.298e+02/2.296e+02, rel reduction: 8.01375e-04, gain ratio: 3.11727e+01
[2024-07-26 14:11:26.797] [info] LM<sym::Optimize> [iter 11] lambda: 1.000e-08, error prev/linear/new: 2.296e+02/2.296e+02/2.294e+02, rel reduction: 8.08292e-04, gain ratio: 5.40277e+01
[2024-07-26 14:11:26.800] [info] LM<sym::Optimize> [iter 12] lambda: 1.000e-08, error prev/linear/new: 2.294e+02/2.294e+02/2.292e+02, rel reduction: 8.15082e-04, gain ratio: 7.31191e+01
[2024-07-26 14:11:26.802] [info] LM<sym::Optimize> [iter 13] lambda: 1.000e-08, error prev/linear/new: 2.292e+02/2.292e+02/2.290e+02, rel reduction: 8.20049e-04, gain ratio: 8.38013e+01
[2024-07-26 14:11:26.804] [info] LM<sym::Optimize> [iter 14] lambda: 1.000e-08, error prev/linear/new: 2.290e+02/2.290e+02/2.288e+02, rel reduction: 8.23287e-04, gain ratio: 8.85802e+01
[2024-07-26 14:11:26.807] [info] LM<sym::Optimize> [iter 15] lambda: 1.000e-08, error prev/linear/new: 2.288e+02/2.288e+02/2.287e+02, rel reduction: 8.25248e-04, gain ratio: 9.05509e+01
[2024-07-26 14:11:26.809] [info] LM<sym::Optimize> [iter 16] lambda: 1.000e-08, error prev/linear/new: 2.287e+02/2.287e+02/2.285e+02, rel reduction: 8.26344e-04, gain ratio: 9.13594e+01
[2024-07-26 14:11:26.812] [info] LM<sym::Optimize> [iter 17] lambda: 1.000e-08, error prev/linear/new: 2.285e+02/2.285e+02/2.283e+02, rel reduction: 8.26879e-04, gain ratio: 9.16996e+01
[2024-07-26 14:11:26.814] [info] LM<sym::Optimize> [iter 18] lambda: 1.000e-08, error prev/linear/new: 2.283e+02/2.283e+02/2.281e+02, rel reduction: 8.27056e-04, gain ratio: 9.18459e+01
[2024-07-26 14:11:26.816] [info] LM<sym::Optimize> [iter 19] lambda: 1.000e-08, error prev/linear/new: 2.281e+02/2.281e+02/2.279e+02, rel reduction: 8.27005e-04, gain ratio: 9.19070e+01
[2024-07-26 14:11:26.819] [info] LM<sym::Optimize> [iter 20] lambda: 1.000e-08, error prev/linear/new: 2.279e+02/2.279e+02/2.277e+02, rel reduction: 8.26812e-04, gain ratio: 9.19280e+01
[2024-07-26 14:11:26.823] [info] LM<sym::Optimize> [iter 21] lambda: 1.000e-08, error prev/linear/new: 2.277e+02/2.277e+02/2.275e+02, rel reduction: 8.26528e-04, gain ratio: 9.19285e+01
[2024-07-26 14:11:26.825] [info] LM<sym::Optimize> [iter 22] lambda: 1.000e-08, error prev/linear/new: 2.275e+02/2.275e+02/2.273e+02, rel reduction: 8.26187e-04, gain ratio: 9.19178e+01
[2024-07-26 14:11:26.828] [info] LM<sym::Optimize> [iter 23] lambda: 1.000e-08, error prev/linear/new: 2.273e+02/2.273e+02/2.272e+02, rel reduction: 8.25811e-04, gain ratio: 9.19008e+01
[2024-07-26 14:11:26.830] [info] LM<sym::Optimize> [iter 24] lambda: 1.000e-08, error prev/linear/new: 2.272e+02/2.271e+02/2.270e+02, rel reduction: 8.25412e-04, gain ratio: 9.18800e+01
[2024-07-26 14:11:26.833] [info] LM<sym::Optimize> [iter 25] lambda: 1.000e-08, error prev/linear/new: 2.270e+02/2.270e+02/2.268e+02, rel reduction: 8.24998e-04, gain ratio: 9.18569e+01
[2024-07-26 14:11:26.835] [info] LM<sym::Optimize> [iter 26] lambda: 1.000e-08, error prev/linear/new: 2.268e+02/2.268e+02/2.266e+02, rel reduction: 8.24576e-04, gain ratio: 9.18325e+01
[2024-07-26 14:11:26.838] [info] LM<sym::Optimize> [iter 27] lambda: 1.000e-08, error prev/linear/new: 2.266e+02/2.266e+02/2.264e+02, rel reduction: 8.24149e-04, gain ratio: 9.18072e+01
[2024-07-26 14:11:26.840] [info] LM<sym::Optimize> [iter 28] lambda: 1.000e-08, error prev/linear/new: 2.264e+02/2.264e+02/2.262e+02, rel reduction: 8.23719e-04, gain ratio: 9.17814e+01
[2024-07-26 14:11:26.843] [info] LM<sym::Optimize> [iter 29] lambda: 1.000e-08, error prev/linear/new: 2.262e+02/2.262e+02/2.260e+02, rel reduction: 8.23287e-04, gain ratio: 9.17554e+01
[2024-07-26 14:11:26.845] [info] LM<sym::Optimize> [iter 30] lambda: 1.000e-08, error prev/linear/new: 2.260e+02/2.260e+02/2.258e+02, rel reduction: 8.22854e-04, gain ratio: 9.17293e+01
[2024-07-26 14:11:26.848] [info] LM<sym::Optimize> [iter 31] lambda: 1.000e-08, error prev/linear/new: 2.258e+02/2.258e+02/2.257e+02, rel reduction: 8.22422e-04, gain ratio: 9.17031e+01
[2024-07-26 14:11:26.850] [info] LM<sym::Optimize> [iter 32] lambda: 1.000e-08, error prev/linear/new: 2.257e+02/2.257e+02/2.255e+02, rel reduction: 8.21989e-04, gain ratio: 9.16769e+01
[2024-07-26 14:11:26.852] [info] LM<sym::Optimize> [iter 33] lambda: 1.000e-08, error prev/linear/new: 2.255e+02/2.255e+02/2.253e+02, rel reduction: 8.21558e-04, gain ratio: 9.16508e+01
[2024-07-26 14:11:26.855] [info] LM<sym::Optimize> [iter 34] lambda: 1.000e-08, error prev/linear/new: 2.253e+02/2.253e+02/2.251e+02, rel reduction: 8.21127e-04, gain ratio: 9.16248e+01
[2024-07-26 14:11:26.858] [info] LM<sym::Optimize> [iter 35] lambda: 1.000e-08, error prev/linear/new: 2.251e+02/2.251e+02/2.249e+02, rel reduction: 8.20697e-04, gain ratio: 9.15989e+01
[2024-07-26 14:11:26.860] [info] LM<sym::Optimize> [iter 36] lambda: 1.000e-08, error prev/linear/new: 2.249e+02/2.249e+02/2.247e+02, rel reduction: 8.20268e-04, gain ratio: 9.15732e+01
[2024-07-26 14:11:26.862] [info] LM<sym::Optimize> [iter 37] lambda: 1.000e-08, error prev/linear/new: 2.247e+02/2.247e+02/2.245e+02, rel reduction: 8.19841e-04, gain ratio: 9.15475e+01
[2024-07-26 14:11:26.865] [info] LM<sym::Optimize> [iter 38] lambda: 1.000e-08, error prev/linear/new: 2.245e+02/2.245e+02/2.244e+02, rel reduction: 8.19414e-04, gain ratio: 9.15220e+01
[2024-07-26 14:11:26.867] [info] LM<sym::Optimize> [iter 39] lambda: 1.000e-08, error prev/linear/new: 2.244e+02/2.244e+02/2.242e+02, rel reduction: 8.18989e-04, gain ratio: 9.14966e+01
[2024-07-26 14:11:26.870] [info] LM<sym::Optimize> [iter 40] lambda: 1.000e-08, error prev/linear/new: 2.242e+02/2.242e+02/2.240e+02, rel reduction: 8.18565e-04, gain ratio: 9.14713e+01
[2024-07-26 14:11:26.873] [info] LM<sym::Optimize> [iter 41] lambda: 1.000e-08, error prev/linear/new: 2.240e+02/2.240e+02/2.238e+02, rel reduction: 8.18142e-04, gain ratio: 9.14462e+01
[2024-07-26 14:11:26.875] [info] LM<sym::Optimize> [iter 42] lambda: 1.000e-08, error prev/linear/new: 2.238e+02/2.238e+02/2.236e+02, rel reduction: 8.17720e-04, gain ratio: 9.14212e+01
[2024-07-26 14:11:26.877] [info] LM<sym::Optimize> [iter 43] lambda: 1.000e-08, error prev/linear/new: 2.236e+02/2.236e+02/2.234e+02, rel reduction: 8.17299e-04, gain ratio: 9.13964e+01
[2024-07-26 14:11:26.880] [info] LM<sym::Optimize> [iter 44] lambda: 1.000e-08, error prev/linear/new: 2.234e+02/2.234e+02/2.233e+02, rel reduction: 8.16880e-04, gain ratio: 9.13717e+01
[2024-07-26 14:11:26.882] [info] LM<sym::Optimize> [iter 45] lambda: 1.000e-08, error prev/linear/new: 2.233e+02/2.233e+02/2.231e+02, rel reduction: 8.16462e-04, gain ratio: 9.13471e+01
[2024-07-26 14:11:26.884] [info] LM<sym::Optimize> [iter 46] lambda: 1.000e-08, error prev/linear/new: 2.231e+02/2.231e+02/2.229e+02, rel reduction: 8.16045e-04, gain ratio: 9.13227e+01
[2024-07-26 14:11:26.887] [info] LM<sym::Optimize> [iter 47] lambda: 1.000e-08, error prev/linear/new: 2.229e+02/2.229e+02/2.227e+02, rel reduction: 8.15630e-04, gain ratio: 9.12984e+01
[2024-07-26 14:11:26.890] [info] LM<sym::Optimize> [iter 48] lambda: 1.000e-08, error prev/linear/new: 2.227e+02/2.227e+02/2.225e+02, rel reduction: 8.15216e-04, gain ratio: 9.12743e+01
[2024-07-26 14:11:26.892] [info] LM<sym::Optimize> [iter 49] lambda: 1.000e-08, error prev/linear/new: 2.225e+02/2.225e+02/2.224e+02, rel reduction: 8.14803e-04, gain ratio: 9.12504e+01
[2024-07-26 14:11:26.892] [warning] LM<sym::Optimize> Optimization finished with status: HIT_ITERATION_LIMIT
The combination of 1) the fact that you mentioned this used to use GN, and presumably worked ok there and 2) lots of traces like that second one, where lambda monotonically decreases to a very small value also make me think your initial lambda should maybe be much smaller than 1, e.g. 1e-8, which would also make SymForce converge faster. This may also be a reason why you saw dynamic lambda doing worse, because one thing it doesn't do is decrease lambda as quickly.
Your points 2 and 4 indicate that the problem is extremely ill-conditioned, and the hessian is either singular or has a very high condition number.
3 might make sense given this - the dense cholesky solver SymForce uses is Eigen's LDLT, which has pivoting, which may make it more stable in the face of the hessian being extremely ill-conditioned.
When you say this is VINS-MONO, is it the default implementation https://github.com/HKUST-Aerial-Robotics/VINS-Mono, does it have modifications, is it another implementation, etc? Partly I'm wondering whether your initializations are already very good, and the optimizations you're doing here are making things much worse than just the initialization, and what it would look like if the optimization just did nothing
Here's my further test result, all the test is runing on the euroc MH_05_difficult dataset, using Symforce with DYNAMIC lambda parameter. 0727.zip
To facilitate your understanding, I make a table:
And here's some new conclusion:
Partly I'm wondering whether your initializations are already very good
Both Ceres and Symforce use the same initilization result, I didn't modify the initiation part of VINS-MONO.
If you need , I can provide modified VINS-mono Keep me updated.
To be honest, it's unlikely that I'm going to have time to look into this soon. Assuming you're talking about a fork of VINS-Mono, running the way they describe in their README for EuRoC on MH_05_difficult with nothing else involved, yeah it seems like a link to the code should be enough to reproduce, that would be helpful if I or someone else has time to look at some point
Leave your email or any other contact information plz, or you can contact me by [zhangda12138@gmail.com],then I could send you the code.
It's going to be much easier if you make a fork of VINS-Mono, and link to your fork here. If for some reason you don't want your code to be publicly accessible, you can create a private repo and invite my GitHub account.
https://github.com/wxliu1/VINS-Mono-NotedByLWX Here's the code
Hi,
I replaced Ceres with Symforce in a graph-based VSLAM backend, result in a bad accuracy. Here are the details:
Screenshots
![image](https://github.com/user-attachments/assets/751111b7-7451-487e-a2f6-550fca8a1bcd)
Environment
Additional context