Error from Poisson lognormal fitting

[ethanwhite]

When I run the current code I get the following error when it tries to fit the Poisson lognormal using Scipy 0.14.0 and the current master for macroecotools:

In [8]: bbs, Site 11001, S=42, N=436
---------------------------------------------------------------------------
TypeError                                 Traceback (most recent call last)
<ipython-input-8-8efba411c155> in <module>()
----> 1 __pyfile = open('''/tmp/py25186Ew0''');exec(compile(__pyfile.read(), '''/home/ethan/Dropbox/Research/SADComparison/sad-comparisons.py''', 'exec'));__pyfile.close()

/home/ethan/Dropbox/Research/SADComparison/sad-comparisons.py in <module>()
    142     raw_data = import_abundance(datafile) # Import data
    143 
--> 144     model_comparisons(raw_data, dataset_name, data_dir, cutoff = 9) # Run analyses on data

/home/ethan/Dropbox/Research/SADComparison/sad-comparisons.py in model_comparisons(raw_data, dataset_name, data_dir, cutoff)
     71 
     72             # Poisson lognormal
---> 73             mu, sigma = md.pln_solver(obsabundance)
     74             L_pln = md.pln_ll(obsabundance, mu,sigma) # Log-likelihood of Poisson lognormal
     75 

/usr/local/lib/python2.7/dist-packages/macroecotools-0.2-py2.7.egg/macroeco_distributions/macroeco_distributions.pyc in pln_solver(ab, lower_trunc)
    436     def pln_func(x):
    437         return -pln_ll(ab, x[0], x[1], lower_trunc, full_output = 1)
--> 438     mu, sigma = optimize.fmin(pln_func, x0 = [mu0, sig0], disp = 0)
    439     return mu, sigma
    440 

/usr/local/lib/python2.7/dist-packages/scipy/optimize/optimize.pyc in fmin(func, x0, args, xtol, ftol, maxiter, maxfun, full_output, disp, retall, callback)
    371             'return_all': retall}
    372 
--> 373     res = _minimize_neldermead(func, x0, args, callback=callback, **opts)
    374     if full_output:
    375         retlist = res['x'], res['fun'], res['nit'], res['nfev'], res['status']

/usr/local/lib/python2.7/dist-packages/scipy/optimize/optimize.pyc in _minimize_neldermead(func, x0, args, callback, xtol, ftol, maxiter, maxfev, disp, return_all, **unknown_options)
    436     if retall:
    437         allvecs = [sim[0]]
--> 438     fsim[0] = func(x0)
    439     nonzdelt = 0.05
    440     zdelt = 0.00025

/usr/local/lib/python2.7/dist-packages/scipy/optimize/optimize.pyc in function_wrapper(*wrapper_args)
    279     def function_wrapper(*wrapper_args):
    280         ncalls[0] += 1
--> 281         return function(*(wrapper_args + args))
    282 
    283     return ncalls, function_wrapper

/usr/local/lib/python2.7/dist-packages/macroecotools-0.2-py2.7.egg/macroeco_distributions/macroeco_distributions.pyc in pln_func(x)
    435     sig0 = std(log(ab))
    436     def pln_func(x):
--> 437         return -pln_ll(ab, x[0], x[1], lower_trunc, full_output = 1)
    438     mu, sigma = optimize.fmin(pln_func, x0 = [mu0, sig0], disp = 0)
    439     return mu, sigma

/usr/local/lib/python2.7/dist-packages/macroecotools-0.2-py2.7.egg/macroeco_distributions/macroeco_distributions.pyc in pln_ll(x, mu, sigma, lower_trunc, full_output)
    353     observed_abund_vals = cts[1][cts[0] != 0]
    354     counts = cts[0][cts[0] != 0]
--> 355     plik = pln.logpmf(observed_abund_vals, mu, sigma, lower_trunc, full_output = full_output)
    356     lik_list = np.array([], dtype = float)
    357     for i, count in enumerate(counts):

/usr/local/lib/python2.7/dist-packages/scipy/stats/_distn_infrastructure.pyc in logpmf(self, k, *args, **kwds)
   2775 
   2776         """
-> 2777         args, loc, _ = self._parse_args(*args, **kwds)
   2778         k, loc = map(asarray, (k, loc))
   2779         args = tuple(map(asarray, args))

TypeError: _parse_args() got an unexpected keyword argument 'full_output'

[ethanwhite]

@embaldridge are you seeing this as well?

[embaldridge]

Yup. It seems to have popped up all of a sudden.

[evaluate sad-comparisons.py] bbs, Site 4002, S=35, N=816 Traceback (most recent call last): File "/usr/lib/wingide-101-4.1/src/debug/tserver/_sandbox.py", line 146, in File "/usr/lib/wingide-101-4.1/src/debug/tserver/_sandbox.py", line 75, in model_comparisons File "/usr/local/lib/python2.7/dist-packages/macroecotools-0.2-py2.7.egg/macroeco_distributions/macroeco_distributions.py", line 438, in pln_solver File "/usr/lib/python2.7/dist-packages/scipy/optimize/optimize.py", line 373, in fmin res = _minimize_neldermead(func, x0, args, callback=callback, _opts) File "/usr/lib/python2.7/dist-packages/scipy/optimize/optimize.py", line 438, in _minimize_neldermead fsim[0] = func(x0) File "/usr/lib/python2.7/dist-packages/scipy/optimize/optimize.py", line 281, in functionwrapper return function((wrapper_args + args)) File "/usr/local/lib/python2.7/dist-packages/macroecotools-0.2-py2.7.egg/macroeco_distributions/macroeco_distributions.py", line 437, in pln_func File "/usr/local/lib/python2.7/dist-packages/macroecotools-0.2-py2.7.egg/macroeco_distributions/macroeco_distributions.py", line 355, in plnll File "/usr/lib/python2.7/dist-packages/scipy/stats/distributions.py", line 6595, in logpmf args, loc, = self._parse_args(_args, _kwds) TypeError: _parse_args() got an unexpected keyword argument 'full_output'

[ethanwhite]

@rueuntal would you mind taking a peak at this is you have a few minutes?

[rueuntal]

Does it not work for all sites or specific sites?

@embaldridge and @ethanwhite , could either of you post or send me an abundance list where it doesn't work?

[embaldridge]

Coming your way, @rueuntal.

[rueuntal]

I ran

mu, sigma = md.pln_solver(ab)
L_pln = md.pln_ll(ab, mu, sigma)

with the list of abundances @embaldridge sent and it worked fine on Zoe. Is that where the problem occurs for you folks?

[embaldridge]

I think so.

[rueuntal]

Hmmm that's a bit strange. @embaldridge are you running the code on one of the servers? Which one are you using?

[rueuntal]

I tried the little piece of test code on all three servers, and Wash flagged the same error as above while the other two were fine. So I'm guessing that it's likely the result of version differences in some of the modules called by macroeco_distributions (most probably scipy).

[embaldridge]

I've been developing the code on the Laptop of Science. I'm running version 0.13.3 of scipy, if that helps.

[rueuntal]

I've removed all instances of "full_output" from the module and that seems to have fixed the problem. However I'm still not sure what was causing Scipy to fail in the first place.

[embaldridge]

All better, closing this issue.