nans in the objective function? lmfit 0.9.5 vs 0.9.3

petigura commented 7 years ago

Hi, Samuel.

I installed specmatch-emp onto my groups cluster, and when running, I got the following error message. I traced the problem down to lmfit 0.9.5 (cluster) vs 0.9.3 (my machine). There was a change in how lmfit handles nans. Could you look into why we're getting nans at the linear combination stage. I don't think this should happen.

Erik

[specmatch-emp]$ smemp specmatch /mir3/iodfitsdb/rj201.587.fits -o /mir3/iodfitsdb/specmatchemp-workdir/test
Reading library from /o/petigura/.specmatchemp/library.h5
Shifting spectrum

Shifting spectrum rj201.587
Attempting shift to spectrum NSO, median cross-correlation peak = 2.69
Attempting shift to spectrum 123239, median cross-correlation peak = 2.79
Attempting shift to spectrum 222368, median cross-correlation peak = 1.76
Attempting shift to spectrum 216899, median cross-correlation peak = 1.14
Best reference for shifting: 123239
Shifting entire spectrum
Matching spectrum
Matching region (5000, 5100)
Matching region (5100, 5200)
Matching region (5200, 5300)
Matching region (5300, 5400)
Matching region (5400, 5500)
Matching region (5500, 5600)
Matching region (5600, 5700)
Matching region (5700, 5800)
Matching region (5800, 5900)
Creating linear combinations
Linear combinations in region (5000, 5100)
Traceback (most recent call last):
  File "/mir3/petigura/anaconda/bin/smemp", line 11, in <module>
    load_entry_point('SpecMatch-Emp==0.2', 'console_scripts', 'smemp')()
  File "build/bdist.macosx-10.6-x86_64/egg/specmatchemp/cli.py", line 129, in main
  File "build/bdist.macosx-10.6-x86_64/egg/specmatchemp/cli.py", line 17, in specmatch_spectrum
  File "build/bdist.macosx-10.6-x86_64/egg/specmatchemp/core.py", line 62, in specmatch_spectrum
  File "build/bdist.macosx-10.6-x86_64/egg/specmatchemp/specmatch.py", line 312, in lincomb
  File "build/bdist.macosx-10.6-x86_64/egg/specmatchemp/match.py", line 425, in best_fit
  File "/mir3/petigura/anaconda/lib/python2.7/site-packages/lmfit/minimizer.py", line 1543, in minimize
    return fitter.minimize(method=method)
  File "/mir3/petigura/anaconda/lib/python2.7/site-packages/lmfit/minimizer.py", line 1242, in minimize
    return function(**kwargs)
  File "/mir3/petigura/anaconda/lib/python2.7/site-packages/lmfit/minimizer.py", line 577, in scalar_minimize
    ret = scipy_minimize(self.penalty, vars, **fmin_kws)
  File "/mir3/petigura/anaconda/lib/python2.7/site-packages/scipy/optimize/_minimize.py", line 438, in minimize
    return _minimize_neldermead(fun, x0, args, callback, **options)
  File "/mir3/petigura/anaconda/lib/python2.7/site-packages/scipy/optimize/optimize.py", line 517, in _minimize_neldermead
    fsim[k] = func(sim[k])
  File "/mir3/petigura/anaconda/lib/python2.7/site-packages/scipy/optimize/optimize.py", line 292, in function_wrapper
    return function(*(wrapper_args + args))
  File "/mir3/petigura/anaconda/lib/python2.7/site-packages/lmfit/minimizer.py", line 425, in penalty
    r = self.__residual(fvars)
  File "/mir3/petigura/anaconda/lib/python2.7/site-packages/lmfit/minimizer.py", line 371, in __residual
    out = _nan_policy(out, nan_policy=self.nan_policy)
  File "/mir3/petigura/anaconda/lib/python2.7/site-packages/lmfit/minimizer.py", line 1432, in _nan_policy
    raise ValueError("The input contains nan values")
ValueError: The input contains nan values

samuelyeewl commented 7 years ago

Sorry for the delay - it took me a while to pin down the problem as it's something that appears only when using python 2. Just had to add a single character to ensure the lincomb coefficients were interpreted as floats and not ints. Fixed in 4ea2015.

petigura commented 7 years ago

No worries :-). Thanks for making the fix. I can verify that this works on my machine with lmfit=0.9.5

samuelyeewl / specmatch-emp

nans in the objective function? lmfit 0.9.5 vs 0.9.3 #22