Spotless crash following re-gridding

[IanHeywood]

Modified grid step and resuming spotless following the fringe explosion as advised:

pfb grid -o pfb_GCX17r2 -ntd 8 -nvt 4 -fov 3 -srf 2.5 -rob -1.0 -ldir pfb_logs --nband 8 --fits-cubes --l2reweight-dof 3.5 --use-best-model

pfb spotless -o pfb_GCX17r2 -nb 8 -ntd 8 -nvt 4 -p I --rmsfactor 0.65 -niter 10 --l1reweight-from 5 -pdtol 2e-4 -pdmaxit 450 --tol 5e-4 -ldir pfb_logs

Spotless doesn't get off the ground:

INFO      14:55:05 - SPOTLESS           | Combining slices into cubes
INFO      14:57:01 - SPOTLESS           | Finding spectral norm of Hessian approximation
INFO      15:02:47 - PM                 | Success, converged after 29 iterations. beta = 2.001e+01
INFO      15:02:47 - SPOTLESS           | Setting up dictionary
Traceback (most recent call last):
  File "/home/ianh/venv/pfb/bin/pfb", line 33, in <module>
    sys.exit(load_entry_point('pfb-clean', 'console_scripts', 'pfb')())
  File "/home/ianh/venv/pfb/lib/python3.9/site-packages/click/core.py", line 1157, in __call__
    return self.main(*args, **kwargs)
  File "/home/ianh/venv/pfb/lib/python3.9/site-packages/click/core.py", line 1078, in main
    rv = self.invoke(ctx)
  File "/home/ianh/venv/pfb/lib/python3.9/site-packages/click/core.py", line 1688, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
  File "/home/ianh/venv/pfb/lib/python3.9/site-packages/click/core.py", line 1434, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "/home/ianh/venv/pfb/lib/python3.9/site-packages/click/core.py", line 783, in invoke
    return __callback(*args, **kwargs)
  File "/home/ianh/Software/pfb-clean/pfb/workers/spotless.py", line 54, in spotless
    return _spotless(**opts)
  File "/home/ianh/Software/pfb-clean/pfb/workers/spotless.py", line 217, in _spotless
    GaussPar = fitcleanbeam(psf_mfs[None], level=0.5, pixsize=1.0)[0]
  File "/home/ianh/Software/pfb-clean/pfb/utils/misc.py", line 566, in fitcleanbeam
    p, _ = curve_fit(func, xy, psfv, p0=(emaj0, emin0, 0.0),
  File "/home/ianh/venv/pfb/lib/python3.9/site-packages/scipy/optimize/_minpack_py.py", line 982, in curve_fit
    raise RuntimeError("Optimal parameters not found: " + errmsg)
RuntimeError: Optimal parameters not found: Number of calls to function has reached maxfev = 2000.

[landmanbester]

Thanks for reporting @IanHeywood. This is an issue with the PSF fitting, it seems to need more than the default of 2000 iterations. Note to self - pass in the gradient!

[landmanbester]

Weird given that the PSF looks ok

[landmanbester]

Alright, I've switched from curve_fit to using fmin_l_bfgs_b with a value_and_grad from jax so this shouldn't happen anymore. Please test again and report here if the error persists