numpy deprecation warnings

[jairideout]

@wasade noted in #1847 that there are numpy deprecation warnings being raised with numpy 1.9.1:

/usr/local/lib/python2.7/dist-packages/numpy/core/fromnumeric.py:2499: VisibleDeprecationWarning: `rank` is deprecated; use the `ndim` attribute or function instead. To find the rank of a matrix see `numpy.linalg.matrix_rank`.
  VisibleDeprecationWarning)
Error in validate_mapping_file.py: option -m: file does not exist: 'map.tsv'

If you need help with QIIME, see:
http://help.qiime.org

I think these warnings are coming from PyCogent beta diversity code. It's strange that the warnings are being raised in scripts that aren't related to beta diversty (e.g., validate_mapping_file.py). We should figure out how to suppress these warnings, or at least suppress them for scripts that aren't related to beta diversity.

Note: I'm not able to reproduce this locally with numpy 1.9.1.

[wasade]

Ya think the calls are implicit based in grepping qiime and skbio code bases. Issue is with numpy.rank I believe On Jan 11, 2015 6:51 PM, "Jai Ram Rideout" notifications@github.com wrote:

@wasade https://github.com/wasade noted in #1847 https://github.com/biocore/qiime/issues/1847 that there are numpy deprecation warnings being raised with numpy 1.9.1:

/usr/local/lib/python2.7/dist-packages/numpy/core/fromnumeric.py:2499: VisibleDeprecationWarning: rank is deprecated; use the ndim attribute or function instead. To find the rank of a matrix see numpy.linalg.matrix_rank. VisibleDeprecationWarning) Error in validate_mapping_file.py: option -m: file does not exist: 'map.tsv'

If you need help with QIIME, see:http://help.qiime.org

I think these warnings are coming from PyCogent beta diversity code. It's strange that the warnings are being raised in scripts that aren't related to beta diversty (e.g., validate_mapping_file.py). We should figure out how to suppress these warnings, or at least suppress them for scripts that aren't related to beta diversity.

Note: I'm not able to reproduce this locally with numpy 1.9.1.

— Reply to this email directly or view it on GitHub https://github.com/biocore/qiime/issues/1850.

[jairideout]

I think cogent/maths/distance_transform.py is the offending file (e.g., see this line).

[wasade]

That would do it, good find... On Jan 11, 2015 9:06 PM, "Jai Ram Rideout" notifications@github.com wrote:

I think cogent/maths/distance_transform.py is the offending file (e.g., see this line https://github.com/pycogent/pycogent/blob/master/cogent/maths/distance_transform.py#L188 ).

— Reply to this email directly or view it on GitHub https://github.com/biocore/qiime/issues/1850#issuecomment-69508957.

[jairideout]

@wasade after upgrading to QIIME 1.9.0-rc2 on your test EC2 instance, I'm not seeing these deprecation warnings anymore. I also tested on our 1.9.0-rc2 base install AMI and I can't reproduce the warnings there either. Both AMIs are using numpy 1.9.1. Closing for now, please reopen if the issue resurfaces.

[ElDeveloper]

@jairideout Andrew reported this problem with rc2, so I think this is still an issue:

https://groups.google.com/forum/#!topic/qiime-forum/e0l0LIljNs0

It seems that upgrading scipy fixes the problem.

[jairideout]

Thanks! There isn't much we can do about the issue if it's coming from older versions of scipy. Can we close this and recommend that users upgrade their scipy version if they're getting these warnings? Alternatively, we could update setup.py to require scipy >= 0.15.0, though that seems pretty restrictive...

[wasade]

Awesome, thank you! On Jan 13, 2015 8:21 PM, "Jai Ram Rideout" notifications@github.com wrote:

@wasade https://github.com/wasade after upgrading to QIIME 1.9.0-rc2 on your test EC2 instance, I'm not seeing these deprecation warnings anymore. I also tested on our 1.9.0-rc2 base install AMI and I can't reproduce the warnings there either. Both AMIs are using numpy 1.9.1. Closing for now, please reopen if the issue resurfaces.

— Reply to this email directly or view it on GitHub https://github.com/biocore/qiime/issues/1850#issuecomment-69801505.

[wasade]

I thought the instance was using scipy 0.13.3? Or I might be misremembering On Jan 13, 2015 8:23 PM, "Jai Ram Rideout" notifications@github.com wrote:

Thanks! There isn't much we can do about the issue if it's coming from older versions of scipy. Can we close this and recommend that users upgrade their scipy version if they're getting these warnings? Alternatively, we could update setup.py to require scipy >= 0.15.0, though that seems pretty restrictive...

— Reply to this email directly or view it on GitHub https://github.com/biocore/qiime/issues/1850#issuecomment-69802703.

[jairideout]

@ElDeveloper and I chatted about this offline and decided to update setup.py to require scipy >= 0.15.0. Since QIIME requires newer versions of numpy that raise this deprecation warning, it's probably safest to require newer versions of scipy to avoid user confusion. I'll submit a pull request to fix this.

[jairideout]

@wasade the instance was using that version of scipy, but when I upgraded it to 1.9.0-rc2, it grabbed scipy 0.15.0, which was why the warnings disappeared.

[wasade]

+1 On Jan 13, 2015 8:36 PM, "Jai Ram Rideout" notifications@github.com wrote:

@ElDeveloper https://github.com/ElDeveloper and I chatted about this offline and decided to update setup.py to require scipy >= 0.15.0. Since QIIME requires newer versions of numpy that raise this deprecation warning, it's probably safest to require newer versions of scipy to avoid user confusion. I'll submit a pull request to fix this.

— Reply to this email directly or view it on GitHub https://github.com/biocore/qiime/issues/1850#issuecomment-69804960.

[alk224]

My problem was I installed scipy from ubuntu repositories which gave me 0.13.3. I upgraded with pip according to zech's instructions (see Yoshiki's post above) and that gave me 0.15 instead.

[jairideout]

Thanks @alk224! When #1854 is merged, this issue shouldn't occur in future QIIME installs because QIIME will require newer versions of scipy.

[jairideout]

Looking into this further, scipy >= 0.14.0 should work (just verified locally). Updated my pull request to relax the version requirement.