when use pwlf, I encounter errors: TypeError: must be real number, not NoneType

[lk1983823]

when use pwlf, I encounter errors: TypeError: must be real number, not NoneType Below is my code: x = np.random.rand(100,1) y = np.random.rand(100,1) myPWLF = pwlf.PiecewiseLinFit(x.reshape(-1), y.reshape(-1)) myPWLF.fit(6) But when I changed another PC, this problem disappears. So I think it may be due to some depend library.

Traceback (most recent call last): File "test.py", line 69, in y_plf, x_plf = model_tm.plf(zkOrder_start, zkOrder_end, y_total, z_fit) File "/media/disk2/sgcc_model_tm2/Model/model_tm.py", line 129, in plf myPWLF.fit(z_fit) File "/home/lukuan/.pyenv/versions/anaconda3-5.1.0/envs/lk3/lib/python3.6/site-packages/pwlf/pwlf.py", line 971, in fit atol=1e-4) File "/home/lukuan/.pyenv/versions/anaconda3-5.1.0/envs/lk3/lib/python3.6/site-packages/scipy/optimize/_differentialevolution.py", line 213, in differential_evolution return solver.solve() File "/home/lukuan/.pyenv/versions/anaconda3-5.1.0/envs/lk3/lib/python3.6/site-packages/scipy/optimize/_differentialevolution.py", line 511, in solve self._calculate_population_energies() File "/home/lukuan/.pyenv/versions/anaconda3-5.1.0/envs/lk3/lib/python3.6/site-packages/scipy/optimize/_differentialevolution.py", line 590, in _calculate_population_energies *self.args) File "/home/lukuan/.pyenv/versions/anaconda3-5.1.0/envs/lk3/lib/python3.6/site-packages/pwlf/pwlf.py", line 670, in fit_with_breaks_opt beta, ssr, rank, s = np.linalg.lstsq(A, self.y_data, rcond=None) File "/home/lukuan/.pyenv/versions/anaconda3-5.1.0/envs/lk3/lib/python3.6/site-packages/numpy/linalg/linalg.py", line 1953, in lstsq 0, work, -1, iwork, 0) TypeError: must be real number, not NoneType

Thank you!

[cjekel]

Can you give me the versions of numpy and pwlf?

pwlf.__version__
np.__version__

[lk1983823]

pwlf.version 0.3.3 np.version 1.14.3

[cjekel]

I'll have to create an environment with numpy==1.14.3 later today. I know numpy<1.14 would break because they changed how rcond is handled.

In the meantime is it possible for you to update numpy to at least numpy>=1.15.4 and see if that fixes your problem?

I think the anaconda command within your environment would be

conda update numpy 

---

Can you run this simple least squares fit? Or do you get the same error?

# https://stackoverflow.com/questions/27128688/how-to-use-least-squares-with-weight-matrix-in-python
import numpy as np
A=np.array([[1,1,1,1],[1,1,1,1],[1,1,1,1],[1,1,1,1],[1,1,0,0]])
B = np.array([1,1,1,1,1])
W = np.array([1,2,3,4,5])
Aw = A * np.sqrt(W[:,np.newaxis])
Bw = B * np.sqrt(W)
X = np.linalg.lstsq(Aw, Bw, rcond=None)

[lk1983823]

Thank you! By upgrading my numpy to 1.15.4, the problem disappears! :+1:

[cjekel]

I'm glad you got it working!

I tried setting up a numpy==0.14.3 environment and couldn't reproduce. I think something was wrong with your numpy build =/

---

Edit. I should note that I was expecting this error to occur from numpy<=0.14. Perhaps I should I add rcond as an optional parameter in the class...

[stklik]

FYI, I keep seeing the same error with the pwlf==0.4.2 and numpy==1.16.2.

Running the code from above produces

---------------------------------------------------------------------------
TypeError                                 Traceback (most recent call last)
<ipython-input-2-e4acef74a3ed> in <module>()
      5 Aw = A * np.sqrt(W[:,np.newaxis])
      6 Bw = B * np.sqrt(W)
----> 7 X = np.linalg.lstsq(Aw, Bw, rcond=None)

/opt/conda/lib/python3.6/site-packages/numpy/linalg/linalg.py in lstsq(a, b, rcond)
   1913         work = zeros((lwork,), t)
   1914         results = lapack_routine(m, n, n_rhs, a, m, bstar, ldb, s, rcond,
-> 1915                                  0, work, -1, iwork, 0)
   1916         lwork = int(work[0])
   1917         work = zeros((lwork,), t)

TypeError: must be real number, not NoneType

changing rcond=-1 a suggested here makes it work again.

[cjekel]

@stklik Are you absolute sure that you are running numpy==1.16.2? If so, you should raise an issue on numpy that recommended default of rcond=None does not work with all numpy backends.

Some old discussion on the topic: https://github.com/numpy/numpy/issues/11015 https://github.com/numpy/numpy/pull/9582

---

In terms of pwlf, I could make a parameter in __init__ which sets and option for rcond, allowing for backwards compatibility.

[cjekel]

You can now manually specify rcond when you initialize pwlf. https://github.com/cjekel/piecewise_linear_fit_py/commit/bf67d7d36a1a5009947b464b6381a9d26d9af1bd

I will push pwlf==0.4.3 to pypi shortly.

my_pwlf = pwlf.PiecewiseLinFit(x, y, rcond=-1)

The default is still rcond=None.

[stklik]

so, my pip freeze | grep numpy tells me it's 1.16.2. I'm running inside a Docker image, so I'll try to check if there are any issues with the installations.

I will push pwlf==0.4.3 to pypi shortly.

Thanks, this will certainly solve my problem.

[stklik]

@cjekel After further investigation, I am fairly convinced that my issue has to do with the Jupyter dockerstacks and the fact that they use conda to install numpy while pwlf setup triggers a pip install + some access rights issues. I will investigate and raise it with the people over there if I find out what to do.

PS: Thanks for adding the rcond param, it works for me now, despite "my situation".

--> can be closed from my POV

[cjekel]

Having multiple Python instances is a pain. The pip in your docker setup is probably for the OS's python. I normally run

which pip
which pip3
which pip3.6

to give some idea where the pip files are located. It's possible that one of those is your conda pip. You may have more options... Feel free to email me if you can't seem to get it sorted out.

You can also try

python -m pip install pwlf

to make sure that this Python environment is the same as pip.

If anything rcond adds backwards compatibility for someone stuck on numpy <= 1.14. :)