singular covariance matrices give traceback error for dream/amoeba/newton (Trac #495)

pkienzle commented 5 years ago

Error message:

Error(s) Occurred: Traceback (most recent call last): File "/home/tec/sci/sasview-3.1.2/src/sas/perspectives/fitting/fit_thread.py", line 82, in compute result = map(map_apply, inputs) File "/home/tec/sci/sasview-3.1.2/src/sas/perspectives/fitting/fit_thread.py", line 14, in map_apply return apply(arguments[0], arguments[1:]) File "/home/tec/sci/sasview-3.1.2/src/sas/perspectives/fitting/fit_thread.py", line 11, in map_getattr return getattr(classInstance, classFunc)(args) File "/home/tec/sci/sasview-3.1.2/src/sas/fit/BumpsFitting.py", line 274, in fit result = run_bumps(problem, handler, curr_thread) File "/home/tec/sci/sasview-3.1.2/src/sas/fit/BumpsFitting.py", line 360, in run_bumps 'stderr': fitdriver.stderr() if success else None, File "/usr/local/lib/python2.7/dist-packages/bumps/fitters.py", line 884, in stderr H = lsqerror.hessian(self.problem, self.result[0]) File "/usr/local/lib/python2.7/dist-packages/bumps/lsqerror.py", line 79, in hessian H = nd.Hessian(problem.nllf)(p) File "/usr/local/lib/python2.7/dist-packages/bumps/numdifftools.py", line 1159, in call return self.hessian(x00) File "/usr/local/lib/python2.7/dist-packages/bumps/numdifftools.py", line 1170, in hessian hess = self.hessdiag(x0) File "/usr/local/lib/python2.7/dist-packages/bumps/numdifftools.py", line 1084, in hessdiag return self._partial_der(x00) File "/usr/local/lib/python2.7/dist-packages/bumps/numdifftools.py", line 720, in _partial_der PD[ind] = self._derivative(fun, x0[ind], stepNom[ind]) File "/usr/local/lib/python2.7/dist-packages/bumps/numdifftools.py", line 424, in _derivative der_romb, errors, h2 = self._romb_extrap(der_init, h1) File "/usr/local/lib/python2.7/dist-packages/bumps/numdifftools.py", line 686, in _romb_extrap errest = np.ones(der_init.shape) hout !ValueError: operands could not be broadcast together with shapes (24,) (0,)

Steps: 266 of 1000 chisq: 43.6 ETA: 2s M1.num_pearls: 1.29 | M1.radius: 15.3 | M1.scale: 0.00847
M1.sld_pearl: 1.5e-06 |

Migrated from http://trac.sasview.org/ticket/495

{
    "status": "new",
    "changetime": "2017-10-27T10:38:26",
    "_ts": "2017-10-27 10:38:26.204292+00:00",
    "description": "Error message:\n\n  Error(s) Occurred:\n\tTraceback (most recent call last):\n  File \"/home/tec/sci/sasview-3.1.2/src/sas/perspectives/fitting/fit_thread.py\", line 82, in compute\n    result = map(map_apply, inputs)\n  File \"/home/tec/sci/sasview-3.1.2/src/sas/perspectives/fitting/fit_thread.py\", line 14, in map_apply\n    return apply(arguments[0], arguments[1:])\n  File \"/home/tec/sci/sasview-3.1.2/src/sas/perspectives/fitting/fit_thread.py\", line 11, in map_getattr\n    return  getattr(classInstance, classFunc)(*args)\n  File \"/home/tec/sci/sasview-3.1.2/src/sas/fit/BumpsFitting.py\", line 274, in fit\n    result = run_bumps(problem, handler, curr_thread)\n  File \"/home/tec/sci/sasview-3.1.2/src/sas/fit/BumpsFitting.py\", line 360, in run_bumps\n    'stderr': fitdriver.stderr() if success else None,\n  File \"/usr/local/lib/python2.7/dist-packages/bumps/fitters.py\", line 884, in stderr\n    H = lsqerror.hessian(self.problem, self.result[0])\n  File \"/usr/local/lib/python2.7/dist-packages/bumps/lsqerror.py\", line 79, in hessian\n    H = nd.Hessian(problem.nllf)(p)\n  File \"/usr/local/lib/python2.7/dist-packages/bumps/numdifftools.py\", line 1159, in __call__\n    return self.hessian(x00)\n  File \"/usr/local/lib/python2.7/dist-packages/bumps/numdifftools.py\", line 1170, in hessian\n    hess = self.hessdiag(x0)\n  File \"/usr/local/lib/python2.7/dist-packages/bumps/numdifftools.py\", line 1084, in hessdiag\n    return self._partial_der(x00)\n  File \"/usr/local/lib/python2.7/dist-packages/bumps/numdifftools.py\", line 720, in _partial_der\n    PD[ind] = self._derivative(fun, x0[ind], stepNom[ind])\n  File \"/usr/local/lib/python2.7/dist-packages/bumps/numdifftools.py\", line 424, in _derivative\n    der_romb, errors, h2 = self._romb_extrap(der_init, h1)\n  File \"/usr/local/lib/python2.7/dist-packages/bumps/numdifftools.py\", line 686, in _romb_extrap\n    errest = np.ones(der_init.shape) * hout\n  !ValueError: operands could not be broadcast together with shapes (24,) (0,) \n \n  Steps: 266 of 1000  chisq: 43.6  ETA: 2s\n  M1.num_pearls: 1.29       |       M1.radius: 15.3       |        M1.scale: 0.00847   \n   M1.sld_pearl: 1.5e-06    |  \n\n",
    "reporter": "pkienzle",
    "cc": "",
    "resolution": "",
    "workpackage": "SasView Bug Fixing",
    "time": "2016-01-19T17:44:39",
    "component": "SasView",
    "summary": "singular covariance matrices give traceback error for dream/amoeba/newton",
    "priority": "minor",
    "keywords": "",
    "milestone": "SasView 4.3.0",
    "owner": "",
    "type": "defect"
}

pkienzle commented 5 years ago

Trac update at 2016/01/19 17:46:01:

pkienzle commented:

This error comes about when the covariance matrix is singular. This can happen for several reasons:

(1) correlated parameters. Running DREAM, I see that sld, scale and num_pearls are all correlated in this model.

(2) integer parameters. num_pearls rounds to the nearest integer in the code, which means the partial derivative wrt num_pearls will be zero.

(3) fit bounds. Running DREAM I see that radius is bounded by the top end of the range. When taking the partial derivative at the boundary, the parameter value is clipped to the boundary value, so the partial derivative wrt radius is zero. Freeing radius and letting it rise to its natural value of 100 removes the singularity.

The upstream author of the numerical derivatives package we are using has fixed the problem with singular matrices in the latest release, but this is not yet made its way into sasview. It will still give meaningless parameter uncertainties (probably inf or nan) but the error message will be gone.

pkienzle changed priority from "major" to "minor"

pkienzle commented 5 years ago

Trac update at 2016/01/19 17:52:32:

pkienzle changed attachment from "" to "FitPage1.fitv"
pkienzle commented:

failing fit

butlerpd commented 5 years ago

Trac update at 2016/06/26 17:22:53:

butler commented:

As agreed on fortnightly call of June 21, am moving all minor priority tickets to next release (+1) given the resources and time available to get this release out.

butler changed milestone from "SasView 4.0.0" to "SasView Next Release +1"

butlerpd commented 5 years ago

Trac update at 2017/10/27 10:38:26:

butler commented:

This may now be fixed as er P. K's comment? Needs testing. Also could provide a error trapping routine that warns user that the fit failed to converge properly and not to believe any of the results. Instead try again with different starting parameters and/or change constraints

butler changed milestone from "SasView Next Release +1" to "SasView 4.3.0"

butlerpd commented 4 years ago

@pkienzle @RichardHeenan I guess that the derivative package now captures the singularity error? do we know if this is otherwise stable now and can this ticket be closed? am going to guess that the part about giving better user feedback for inf or NaN would still be applicable but if so suggest a new ticket on refactoring the fitting to return useful information. I will move this to 5.1 for now.

gonzalezma commented 9 months ago

Can we close this?

SasView / sasview

singular covariance matrices give traceback error for dream/amoeba/newton (Trac #495) #628