pip installation not functioning

[kbarnhart]

As pointed out by @tristan-salles in his review of the JOSS submission the pip installation currently does not work.

He reports the following error:

> pip install umami
 Collecting umami
  ERROR: Could not find a version that satisfies the requirement umami (from versions: none)
ERROR: No matching distribution found for umami

[kbarnhart]

@tristan-salles: would you be able to provide me with a few more details of your compute environment and/or installation procedure so that I can address this issue? I just tried to reproduce it on my computer (OSX, Anaconda Python v3.7) and was not able to.

I tried installing umami in my base conda environment and in a new conda environment. Both worked for me.

$ conda create --name test_umami
$ conda activate test_umami
$ conda install pip
$ pip install umami

I was then able to start python and import umami.

Thanks again for your review!

[kbarnhart]

tagging @mcflugen

Also, for reference the pypi page is here

[tristan-salles]

Hi @kbarnhart I did try using a linux ubuntu installation not running conda... this might be the issue. Looking at the installed packages on my current environment I have the following list:

import pip
installed_packages = pip.get_installed_distributions()
installed_packages_list = sorted(["%s==%s" % (i.key, i.version)
     for i in installed_packages])
print(installed_packages_list)

['affine==2.2.2', 'aiohttp==3.5.4', 'alabaster==0.7.12', 'appdirs==1.4.3', 'async-timeout==3.0.1', 'attrs==19.1.0', 'babel==2.7.0', 'backcall==0.1.0', 'bleach==3.1.0', 'cartopy==0.17.0', 'certifi==2019.6.16', 'cftime==1.0.3.4', 'chardet==3.0.4', 'click-plugins==1.1.1', 'click==7.0', 'cligj==0.5.0', 'cycler==0.10.0', 'cython==0.29.13', 'decorator==4.4.0', 'defusedxml==0.6.0', 'descartes==1.1.0', 'docutils==0.14', 'easyprocess==0.2.7', 'entrypoints==0.3', 'escape==0.1', 'fastfunc==0.1.7', 'fillit==0.0.1', 'fillpy==0.0.1', 'geographiclib==1.49', 'geopy==1.20.0', 'glem==0.1', 'h5py==2.9.0', 'idna-ssl==1.1.0', 'idna==2.8', 'imagesize==1.1.0', 'ipykernel==5.1.1', 'ipyparallel==6.2.4', 'ipython-genutils==0.2.0', 'ipython==7.6.1', 'ipywidgets==7.5.0', 'jedi==0.14.0', 'jinja2==2.10.1', 'jsonschema==3.0.1', 'jupyter-client==5.2.4', 'jupyter-console==6.0.0', 'jupyter-core==4.5.0', 'jupyter-server-proxy==1.1.0', 'jupyter==1.0.0', 'kiwisolver==1.1.0', 'lavavu==1.4.0', 'markupsafe==1.1.1', 'matplotlib==3.0.3', 'meshio==2.3.9', 'meshplex==0.11.6', 'mistune==0.8.4', 'mock==3.0.5', 'mpi4py==3.0.2', 'multidict==4.5.2', 'nbconvert==5.5.0', 'nbformat==4.4.0', 'netcdf4==1.5.2', 'notebook==5.7.8', 'numpy==1.17.0', 'packaging==19.0', 'pandas==0.24.2', 'pandocfilters==1.4.2', 'parso==0.5.0', 'pathlib==1.0.1', 'petsc4py==3.11.0', 'pexpect==4.7.0', 'pickleshare==0.7.5', 'pillow==6.1.0', 'pint==0.9', 'pipdate==0.3.2', 'plotly==3.10.0', 'pockets==0.7.2', 'prometheus-client==0.7.1', 'prompt-toolkit==2.0.9', 'ptyprocess==0.6.0', 'pybind11==2.3.0', 'pycpt==0.2.0', 'pygeotools==0.6.0', 'pygments==2.4.2', 'pygmsh==5.0.1', 'pykdtree==1.3.1', 'pyparsing==2.4.0', 'pyrsistent==0.15.2', 'pyshp==2.1.0', 'python-dateutil==2.8.0', 'pytz==2019.1', 'pyvirtualdisplay==0.2.4', 'pyvtk==0.5.18', 'pyzmq==18.0.2', 'qtconsole==4.5.1', 'rasterio==1.0.24', 'requests==2.22.0', 'retrying==1.3.3', 'ruamel.yaml==0.15.98', 'scipy==1.3.1', 'scons==3.0.5', 'seaborn==0.9.0', 'send2trash==1.5.0', 'setuptools==41.0.1', 'shapely==1.6.4.post2', 'simpervisor==0.3', 'six==1.12.0', 'snowballstemmer==1.9.0', 'snuggs==1.4.6', 'sphinx-rtd-theme==0.4.3', 'sphinx==2.1.2', 'sphinxcontrib-applehelp==1.0.1', 'sphinxcontrib-devhelp==1.0.1', 'sphinxcontrib-htmlhelp==1.0.2', 'sphinxcontrib-inlinesyntaxhighlight==0.2', 'sphinxcontrib-jsmath==1.0.1', 'sphinxcontrib-napoleon==0.7', 'sphinxcontrib-qthelp==1.0.2', 'sphinxcontrib-serializinghtml==1.1.3', 'sphinxemoji==0.1.4', 'stripy==1.0.2', 'tabulate==0.8.3', 'terminado==0.8.2', 'testpath==0.4.2', 'tornado==6.0.3', 'traitlets==4.3.2', 'typing-extensions==3.7.4', 'urllib3==1.25.3', 'vtk==8.1.2', 'wcwidth==0.1.7', 'webencodings==0.5.1', 'wheel==0.33.4', 'widgetsnbextension==3.5.0', 'yarl==1.3.0']

After googling a bit for the error message I did find on stackoverflow a potential work around that actually didn't help in my case...

python3 -m pip install <pkg>

If I am the only one having the issue it must be related to my specific machine...

[kbarnhart]

@tristan-salles thanks for the additional information, very helpful.

@mcflugen do you have any recommendations regarding this issue? And/or do you have any thoughts on how we might test installs on systems that have pip only, and no conda install?

(also @mcflugen can you accept my invite to become a contributor to this repo).

[mcflugen]

@kbarnhart I think the problem is that you've built a wheel and uploaded it to PyPI. The version that's on PyPI was built for a specific version of linux. If you do want to distribute a binary package for linux, it should be labeled as manylinux. You can have a look at landlab for an example of how to do this.

However, because umami is pure Python, you should be able to get away with uploading a source distribution, which will then be available for all platforms. You can do this with python setup.py sdist.

[kbarnhart]

@mcflugen quick question...

I thought that the fact that the wheel I uploaded was a none-any wheel meant that it was good for all Linux-based (including OSX) and for Windows. I don't understand why the current wheel works for OSX, some linux, and windows, but not all linux. And thus I don't understand why switching from a none-any wheel to a source distribution will fix things. Do you understand why this might be?

(sidenote, just tested pip install umami on a Windows machine I have access to and my Linux box running Ubuntu 18.04 LTS... both installed OK).

A final note: I tried copying the Landlab manylinux approach when I originally set up the CI for umami, but because umami is pure python all the wheel's had the same name (and so all but the first deploy stages failed due to an upload error from pypi).

[kbarnhart]

@tristan-salles can I ask you to try installing with

pip install umami -i https://pypi.org/simple

That is, explicitly stating to pip install which index-url to get umami from?

[mcflugen]

@tristan-salles Could you also run the following?

$ pip debug

and then, possibly,

$ pip debug -v

This should tell us which tags your pip is looking for and if py3-none-any, which is currently what's on PyPI, is one of those tags.

[mcflugen]

@kbarnhart I don't know why his pip can't find the distribution when most others can. Maybe it's an old version of pip?

[tristan-salles]

Hi @kbarnhart, this gives me the following...

root@300509464e10:/live/share/global/globalData# pip install umami -i https://pypi.org/simple
Collecting umami
ERROR: Could not find a version that satisfies the requirement umami (from versions: none)
ERROR: No matching distribution found for umami

I'm not sure about the reason why pip can't find it... this is not an old version of pip as far as I am aware... pip version: pip 19.2.3...

[tristan-salles]

Hi @mcflugen,

when I ran the pip debug I get:

root@300509464e10:/live/share/global/globalData# pip debug
WARNING: This command is only meant for debugging. Do not use this with automation for parsing and getting these details, since the output and options of this command may change without notice.
pip version: pip 19.2.3 from /usr/local/lib/python3.5/dist-packages/pip (python 3.5)
sys.version: 3.5.3 (default, Sep 27 2018, 17:25:39)
[GCC 6.3.0 20170516]
sys.executable: /usr/bin/python3
sys.getdefaultencoding: utf-8
sys.getfilesystemencoding: utf-8
locale.getpreferredencoding: UTF-8
sys.platform: linux
sys.implementation:
  name: cpython
Compatible tags: 30
  cp35-cp35m-manylinux2010_x86_64
  cp35-cp35m-manylinux1_x86_64
  cp35-cp35m-linux_x86_64
  cp35-abi3-manylinux2010_x86_64
  cp35-abi3-manylinux1_x86_64
  cp35-abi3-linux_x86_64
  cp35-none-manylinux2010_x86_64
  cp35-none-manylinux1_x86_64
  cp35-none-linux_x86_64
  cp34-abi3-manylinux2010_x86_64
  ...
  [First 10 tags shown. Pass --verbose to show all.]

and pip debug -v

root@300509464e10:/live/share/global/globalData# pip debug -v
WARNING: This command is only meant for debugging. Do not use this with automation for parsing and getting these details, since the output and options of this command may change without notice.
pip version: pip 19.2.3 from /usr/local/lib/python3.5/dist-packages/pip (python 3.5)
sys.version: 3.5.3 (default, Sep 27 2018, 17:25:39)
[GCC 6.3.0 20170516]
sys.executable: /usr/bin/python3
sys.getdefaultencoding: utf-8
sys.getfilesystemencoding: utf-8
locale.getpreferredencoding: UTF-8
sys.platform: linux
sys.implementation:
  name: cpython
Compatible tags: 30
  cp35-cp35m-manylinux2010_x86_64
  cp35-cp35m-manylinux1_x86_64
  cp35-cp35m-linux_x86_64
  cp35-abi3-manylinux2010_x86_64
  cp35-abi3-manylinux1_x86_64
  cp35-abi3-linux_x86_64
  cp35-none-manylinux2010_x86_64
  cp35-none-manylinux1_x86_64
  cp35-none-linux_x86_64
  cp34-abi3-manylinux2010_x86_64
  cp34-abi3-manylinux1_x86_64
  cp34-abi3-linux_x86_64
  cp33-abi3-manylinux2010_x86_64
  cp33-abi3-manylinux1_x86_64
  cp33-abi3-linux_x86_64
  cp32-abi3-manylinux2010_x86_64
  cp32-abi3-manylinux1_x86_64
  cp32-abi3-linux_x86_64
  py3-none-manylinux2010_x86_64
  py3-none-manylinux1_x86_64
  py3-none-linux_x86_64
  cp35-none-any
  cp3-none-any
  py35-none-any
  py3-none-any
  py34-none-any
  py33-none-any
  py32-none-any
  py31-none-any
  py30-none-any

This is returning the py3-none-any tag...

Sorry for the pain!

[mcflugen]

@tristan-salles Thank you!

I think the problem is that you have Python 3.5 installed (is that right? From pip debug it appears so). If I do the following, I can reproduce your problem on my Mac,

$ conda create -n py35 python=3.5
$ conda activate py35
$ pip install umami
Collecting umami
  Could not find a version that satisfies the requirement umami (from versions: )
No matching distribution found for umami

However, if I try to install umami within a Python 3.7 environment,

$ conda create -n py37 python=3.7
$ conda activate py37
$ pip install umami

umami installs just fine.

[mcflugen]

@kbarnhart I think the Python version requirement is coming from the python_requires keyword in the setup.py file.

[kbarnhart]

@tristan-salles thanks for your help! And @mcflugen thanks for figuring it out! I'll fix the setup.py and make a new release.

[kbarnhart]

@mcflugen As part of making and testing this new release I've run into an issue that I need your help with.

As Landlab only puts a py3.6 and py3.7 release on PyPI, this means that pip install umami fails for python versions 3.6 and lower. After making the current v0.1.3 release for umami I tested an install with py3.4 and py3.5 (I tried to make a py3.0, py3.1, py3.2, and py3.3 but conda didn't support it).

On both the py3.4 and py3.5 tests I ran into the same problem:

$ conda create -n py34 python=3.4
$ conda activate py34
$ pip install umami
Collecting umami
  Cache entry deserialization failed, entry ignored
  Downloading https://files.pythonhosted.org/packages/38/b7/157fb76582aca42c052f5833fdc33b1fc851c34724c
645a88c08c2d00cd7/umami-0.1.3-py3-none-any.whl
Collecting numpy (from umami)
  Cache entry deserialization failed, entry ignored
  Downloading https://files.pythonhosted.org/packages/db/ec/93ddd4696e9cce0ffb8429516a8ba0d0ee95911cbbadde2d23665b62ad39/numpy-1.16.5.zip (5.1MB)
    100% |████████████████████████████████| 5.1MB 265kB/s
Collecting landlab>=1.10.1 (from umami)
  Cache entry deserialization failed, entry ignored
  Could not find a version that satisfies the requirement landlab>=1.10.1 (from umami) (from
 versions: 0.1.7.macosx-10.5-x86_64, 0.1.6, 0.1.7, 0.1.8, 0.1.9, 0.1.12, 0.1.14, 0.1.15, 0.1.18, 0.1.21,
 0.1.22, 0.1.23, 0.1.24, 0.1.25, 0.1.26, 0.1.27, 0.1.28, 0.1.29, 0.1.33, 0.1.35, 0.1.37, 0.1.38, 0.1.39, 0.1.40,
 0.1.41, 0.2.0, 0.2.1, 0.2.2, 0.2.3, 0.5.0, 1.0.0a0, 1.0.0b0, 1.0.0b1, 1.0.0b3, 1.0.0b4, 1.0.0b6, 1.0.0b8,
 1.0.0b10, 1.0.0b11, 1.0.0b12, 1.0.0, 1.0.1, 1.0.2, 1.0.3, 1.1.0, 1.2.0, 1.3.0, 1.3.1, 1.4.0, 1.5.0, 1.5.1, 1.5.2,
 1.5.3, 1.5.4, 1.9.0b0, 1.9.0b1, 1.9.0b3, 1.9.0b4, 1.9.0b5, 1.9.0)
No matching distribution found for landlab>=1.10.1 (from umami)
You are using pip version 9.0.1, however version 19.2.3 is available.
You should consider upgrading via the 'pip install --upgrade pip' command.

My question: Is this a problem? In theory a user working on py3.5 could download landlab from source and compile, and then get umami from PyPI...

What do you think makes the most sense?

[kbarnhart]

Also, thanks @tristan-salles for helping us identify and diagnose this problem. I'm really glad you found it. 🎉

[tristan-salles]

Hi @kbarnhart & @mcflugen I did try with python 3.8 and the pip package is working as expected...

[mcflugen]

@kbarnhart I think relaxing the python_requires to >=3 for umami is good and probably all that needs to be done. If a user really wants to use an old version of Python they can, as you say, install landlab (and any other packages that aren't available for their version of Python) from source.

We can certainly discuss building landlab for old Python versions but I'm not certain it would be worth it - particularly since one can always create a separate environment for their install.

Its too bad the error message that pip gives isn't more informative.

[kbarnhart]

Thanks for that input @mcflugen. I'm going to close this issue now.