Less efficient grss.fit.FitSimulation.filter_lsq()

[Pachacoti]

After upgrading grss to the latest version, I start to have grss.fit.FitSimulation.filter_lsq() working less efficiently. Using exactly the same dataset of 2003 YH136 below. Older version has:

Iteration       Unweighted RMS      Weighted RMS        Chi-squared     Reduced Chi-squared
1            0.530           0.510           43.144          0.271
2            0.528           0.512           43.537          0.274
3            0.523           0.504           42.245          0.266
4            0.522           0.505           42.338          0.266
5            0.522           0.504           42.225          0.266
6            0.522           0.504           42.226          0.266
Converged without rejecting outliers. Starting outlier rejection now...
7            0.522           0.504           42.234          0.266
8            0.522           0.504           42.250          0.266
9            0.522           0.505           42.266          0.266
10           0.522           0.504           42.230          0.266
11           0.522           0.505           42.310          0.266
12           0.486           0.476           37.638          0.240
13           0.522           0.505           42.292          0.266
14           0.523           0.504           42.242          0.266
15           0.522           0.504           42.234          0.266
Converged after rejecting outliers. Rejected 0 out of 83 optical observations.

The latest version has:

Iteration       Unweighted RMS      Weighted RMS        Chi-squared     Reduced Chi-squared
1            0.531           0.508           42.882          0.270
2            0.536           0.519           44.731          0.281
3            0.542           0.525           45.819          0.288
4            0.533           0.516           44.176          0.278
5            0.524           0.505           42.343          0.266
6            0.912           0.883           129.429             0.814
7            0.709           0.689           78.774          0.495
8            0.578           0.562           52.387          0.329
9            0.526           0.508           42.869          0.270
10           0.553           0.537           47.855          0.301
WARNING: Maximum number of iterations reached without converging.

Though I set different max # of iterations in this test, we can still see that the older algorithm does a better job in finding a converging solution.

[rahil-makadia]

I see the following when I try to fit the orbit for 2003 YH136

Iteration       Unweighted RMS      Weighted RMS        Chi-squared     Reduced Chi-squared
1            0.528           0.506           42.570          0.266
2            0.522           0.503           41.958          0.262
Converged without rejecting outliers. Starting outlier rejection now...
3            0.522           0.503           41.958          0.262
Converged after rejecting outliers. Rejected 0 out of 83 optical observations.

Can you send a file that reproduces your error?

[Pachacoti]

It's basically the same code as your examples, except for adding the following:

init_sol['a2']=1e-15
init_cov = np.append(init_cov, np.zeros((1,6)), axis=0)
init_cov = np.append(init_cov, np.zeros((7,1)), axis=1)
init_cov[-1,-1]=1e-30

[rahil-makadia]

Does the filter converge without the "fake" nongravitational accelerations? If so, then you're probably adding wildly incompatible nongravitational accelerations to the fit... The reason it may not converge now, and it was before is unclear, but it is possible some of the upgrades were previously missing systematics being misattributed to nongravitational acceleration in the fit (allowing it to find some sort of weak convergence previously)...

[Pachacoti]

Yes, it converges way faster if nongrav is not included for the fit.

[rahil-makadia]

Perfect, closing issue then!