ufunc 'isfinite' not supported error

[bluegenes]

Hi folks,

took this out for a spin and am getting this error (using the README example params) - any thoughts?

python ./p-from-scaled-containment.py L=100K k=21 C=0.95 --sccon=0.10605
L       k       conf    Cks     CLow    CHigh   pLow    pHigh
Traceback (most recent call last):
  File "./p-from-scaled-containment.py", line 193, in <module>
    if __name__ == "__main__": main()
  File "./p-from-scaled-containment.py", line 120, in main
    sol1_mpf = newton(f1_mpf, Nm_guess)
  File "/home/ntpierce/miniconda3/envs/mrcc/lib/python3.8/site-packages/scipy/optimize/zeros.py", line 350, in newton
    if np.isclose(p, p1, rtol=rtol, atol=tol):
  File "<__array_function__ internals>", line 5, in isclose
  File "/home/ntpierce/miniconda3/envs/mrcc/lib/python3.8/site-packages/numpy/core/numeric.py", line 2362, in isclose
    xfin = isfinite(x)
TypeError: ufunc 'isfinite' not supported for the input types, and the inputs could not be safely coerced to any supported types according to the casting rule ''safe''

Also, if this could be a version issue for any of: python3, scipy, numpy, mpmath, mmh3, please let me know if there's a specific version or installation method (conda,pip) you recommend :). For reference, I'm currently installing them within an isolated conda env, https://github.com/bluegenes/mutation-rate-ci-calculator/blob/init_pkg/environment.yml.

thx!

[mahmudhera]

Hi, can you please let me know which version of scipy and mpmath do you have? That might help me sort out the issue. I have no problem running this, so maybe I have to try in your versions and see if this really is a version issue.

Thanks!

[bluegenes]

of course, sorry for not providing that initially!

scipy 1.6.2 mpmath 1.2.1

running on Linux x86_64 (Linux bm16 5.4.0-70-generic #78~18.04.1-Ubuntu SMP Sat Mar 20 14:10:07 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux)

If you send me the versions you have installed, I'm happy to troubleshoot on my end!

all package info in my conda virtual environment:

# packages in environment at /home/ntpierce/miniconda3/envs/mrcc:
#
# Name                    Version                   Build  Channel
_libgcc_mutex             0.1                 conda_forge    conda-forge
_openmp_mutex             4.5                       1_gnu    conda-forge
ca-certificates           2020.12.5            ha878542_0    conda-forge
certifi                   2020.12.5        py38h578d9bd_1    conda-forge
ld_impl_linux-64          2.35.1               hea4e1c9_2    conda-forge
libblas                   3.9.0                8_openblas    conda-forge
libcblas                  3.9.0                8_openblas    conda-forge
libffi                    3.3                  h58526e2_2    conda-forge
libgcc-ng                 9.3.0               h2828fa1_18    conda-forge
libgfortran-ng            9.3.0               hff62375_18    conda-forge
libgfortran5              9.3.0               hff62375_18    conda-forge
libgomp                   9.3.0               h2828fa1_18    conda-forge
liblapack                 3.9.0                8_openblas    conda-forge
libopenblas               0.3.12          pthreads_h4812303_1    conda-forge
libstdcxx-ng              9.3.0               h6de172a_18    conda-forge
mmh3                      3.0.0            py38h709712a_0    conda-forge
mpmath                    1.2.1              pyhd8ed1ab_0    conda-forge
ncurses                   6.2                  h58526e2_4    conda-forge
numpy                     1.20.2           py38h9894fe3_0    conda-forge
openssl                   1.1.1k               h7f98852_0    conda-forge
pip                       21.0.1             pyhd8ed1ab_0    conda-forge
python                    3.8.8           hffdb5ce_0_cpython    conda-forge
python_abi                3.8                      1_cp38    conda-forge
readline                  8.0                  he28a2e2_2    conda-forge
scipy                     1.6.2            py38h7b17777_0    conda-forge
setuptools                49.6.0           py38h578d9bd_3    conda-forge
sqlite                    3.35.4               h74cdb3f_0    conda-forge
tk                        8.6.10               hed695b0_1    conda-forge
wheel                     0.36.2             pyhd3deb0d_0    conda-forge
xz                        5.2.5                h516909a_1    conda-forge
zlib                      1.2.11            h516909a_1010    conda-forge

let me know if you need additional info. thanks!!

[mahmudhera]

My apologies Tessa. I was not able to upgrade to your versions (I manually installed them on my system) and therefore, could not recreate the error.

I am using scipy 1.1.0, numpy 1.12.1 and mpmath 1.0.0.

I suspect this arises from using mpmath's arbitrary precision? I could not think of any other way input type could make an issue. If that is the case, defining mpf as follows may solve the issue: mpf = lambda v:float(v).

Let me know of any developments.

[bluegenes]

Using the conda environment defined above, mpf = lambda v:float(v) solves the issue -- it runs and produces the same values from the README! Does this mean it's not using mpmath precision anymore? How much of a problem is this fix?

I also created a conda environment with python 3.6 and your versions of scipy, numpy and mpmath (all package versions below), and everything ran fine without redefining mpf. My concern here is that these versions require python 3.6, which makes things a bit less future-proof.

# packages in environment at /home/ntpierce/miniconda3/envs/mrcc-older:
#
# Name                    Version                   Build  Channel
_libgcc_mutex             0.1                 conda_forge    conda-forge
_openmp_mutex             4.5                       1_gnu    conda-forge
blas                      1.1                    openblas    conda-forge
ca-certificates           2020.12.5            ha878542_0    conda-forge
certifi                   2020.12.5        py36h5fab9bb_1    conda-forge
ld_impl_linux-64          2.35.1               hea4e1c9_2    conda-forge
libffi                    3.3                  h58526e2_2    conda-forge
libgcc-ng                 9.3.0               h2828fa1_19    conda-forge
libgfortran-ng            7.5.0               h14aa051_19    conda-forge
libgfortran4              7.5.0               h14aa051_19    conda-forge
libgomp                   9.3.0               h2828fa1_19    conda-forge
libstdcxx-ng              9.3.0               h6de172a_19    conda-forge
mmh3                      3.0.0            py36hc4f0c31_0    conda-forge
mpmath                    1.0.0                      py_1    conda-forge
ncurses                   6.2                  h58526e2_4    conda-forge
numpy                     1.12.1          py36_blas_openblash1522bff_1001    conda-forge
openblas                  0.3.3             h9ac9557_1001    conda-forge
openssl                   1.1.1k               h7f98852_0    conda-forge
pip                       21.0.1             pyhd8ed1ab_0    conda-forge
python                    3.6.13          hffdb5ce_0_cpython    conda-forge
python_abi                3.6                     1_cp36m    conda-forge
readline                  8.0                  he28a2e2_2    conda-forge
scipy                     1.1.0           py36_blas_openblash1522bff_1202    conda-forge
setuptools                49.6.0           py36h5fab9bb_3    conda-forge
sqlite                    3.35.4               h74cdb3f_0    conda-forge
tk                        8.6.10               hed695b0_1    conda-forge
wheel                     0.36.2             pyhd3deb0d_0    conda-forge
xz                        5.2.5                h516909a_1    conda-forge
zlib                      1.2.11            h516909a_1010    conda-forge

[mahmudhera]

You are absolutely correct Tessa, this means you are not using mpf anymore, simply float.

Even though this seemingly produces the same value as the README, deep down the error associated with the equation we are trying to solve is in the level of 10^-16 using only float, whereas it was around 10^-27 when we used mpf. I am still leaving the issue open, and will be investigating if allowing a larger error would create any issue for our confidence interval.

[bluegenes]

Okay, with this in mind, I'll pin my packages to the earlier versions, so we take advantage of mpf. Ultimately we may still be interested in the results of allowing that larger error, but for now we can avoid!

[bluegenes]

@mahmudhera did you end up investigating how much float vs mpf affected the CI? I'd love to be able to implement some of these within sourmash, but would not be able to pin numpy, scipy, and python to these particular (~old) versions...

[mahmudhera]

I did, actually. Turns out, using float does not really affect the CIs much, only after the 4/5th digit after the decimal point do we see a difference. You can safely make use of floats, I strongly believe.

On Fri, Jan 14, 2022 at 1:01 AM Tessa Pierce Ward @.***> wrote:

@mahmudhera https://github.com/mahmudhera did you end up investigating how much float vs mpf affected the CI? I'd love to be able to implement some of these within sourmash, but would not be able to pin numpy, scipy, and python to these particular (~old) versions...

— Reply to this email directly, view it on GitHub https://github.com/KoslickiLab/mutation-rate-ci-calculator/issues/2#issuecomment-1012421888, or unsubscribe https://github.com/notifications/unsubscribe-auth/AETUQBHAUJLITFZGCA3XNBLUV4OQFANCNFSM42WRTSRA . Triage notifications on the go with GitHub Mobile for iOS https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675 or Android https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub.

You are receiving this because you were mentioned.Message ID: @.***>

[bluegenes]

@mahmudhera awesome, thanks!