search

OxIonics / ionics_fits

Small python fitting library with an emphasis on Atomic Molecular and Optical Physics
Apache License 2.0
1 stars 0 forks source link

Unreasoable error bar for displaced thermal fitting with flaoted `alpha` and `n_bar` #187

Closed pathfinder49 closed 3 days ago

pathfinder49 commented 5 days ago

Brief description

Using LaserFlopTimeDisplacedThermal with floated alpha and n_bar Error bars are unreasonably large in fits.

Details

I'm developing diagnostics for displaced thermal states. Attempting to fit alpha and n_bar to the data in the image below I get a moderatly good fit (shown by the line). (all other parameters are fixed) image

However, logging the fit results (after changing the analysis experiment to show more significant figures) I get fit parameters with unreasonable uncertainties: image These uncertainties are inconsistent with the data being fitted by several orders of magnitude.

Loading the experiment data in a notebook, it appears that the issue only arises when alpha and n_bar are floated together. The actual fitted value is insensitive to these both being floated. However the stated uncertainty is very sensitive. See notebook in https://github.com/OxIonics/analysis/pull/328/commits/2d8675c6a8dfc09c946c2f9c3cb2362c723c4ce6 image

I am surprised by this behavior as I would have expected:

  1. for small n_bar and alpha there should be a large co-variance between n_bar and alpha
  2. n_bar and alpha have small variance
  3. I would have expected the error bar to refelect the small variance rather than the covariance
hartytp commented 3 days ago

So I think the answer here is that the code here is behaving broadly as expected (AFAICT it's reproducing what scipy's curve_fit would do, which is at least one definition of reasonableness!).

As discussed IRL, the issue is that there is an extremely high degree of covariance between the parameters so calculating the variance is numerically challenging. I think this is a case of this not being a reasonable thing to expect to "just work". You /might/ find that the MLE-based NormalFitter does a better job than the default one based on curve_fit, but you'd have to test that.

What I think we should do however is to add the covariance matrix to the fitter's output to make debugging this kind of thing easier. I'll open a separate issue for that.

hartytp commented 3 days ago

Covariances are now exported in https://github.com/OxIonics/ionics_fits/pull/189