Installing pycopy-cpython-upip 1.3.1 through pip fails

[amotl]

Dear Paul,

probably since cpython-upip release 1.3.1 by 6780dd67, we get this error when trying to install it on a vanilla Python 3.7.4 on our workstation.

$ pip install pycopy-cpython-upip
Collecting pycopy-cpython-upip
  Using cached https://files.pythonhosted.org/packages/e0/ac/5ddaf3cd3f86af13702721fff23623fa51382c0ac47082482a74d22f54e1/pycopy-cpython-upip-1.3.1.tar.gz
Collecting pycopy-cpython-cpython-uerrno (from pycopy-cpython-upip)
  ERROR: Could not find a version that satisfies the requirement pycopy-cpython-cpython-uerrno (from pycopy-cpython-upip) (from versions: none)
ERROR: No matching distribution found for pycopy-cpython-cpython-uerrno (from pycopy-cpython-upip)

We found that most of the dependency packages like pycopy-cpython-cpython-uerrno are missing on PyPI altogether. May we humbly ask whether we missed anything about an updated installation procedure which might be required for getting upip installed?

Using cpython-upip==1.2.6 still works well for us, so we explicitly pinned this version for now [1,2].

Thanks in advance for taking a look into this.

With kind regards, Andreas.

[1] https://github.com/hiveeyes/hiveeyes-micropython-firmware/commit/245b886a7dbae6679d2989d6f0a0386b4d8073fc [2] https://github.com/hiveeyes/hiveeyes-micropython-firmware/issues/17

[pfalcon]

Thanks for the report!

pycopy-cpython-cpython-

That double "cpython-" is clearly an error, a result of conflicting update.

Fixing that, I also found that some other cpython-backport modules fell behind the API updates in Pycopy, and fixed them too.

Newly released pycopy-cpython-upip==1.3.3 should work as expected. For reference:

$ pip3 install --no-cache -U --user pycopy-cpython-upip
Collecting pycopy-cpython-upip
  Downloading https://files.pythonhosted.org/packages/16/34/bd3f11cdc85e499c76fbad859e4ed9db900fb13935b670ac695739b3e650/pycopy-cpython-upip-1.3.3.tar.gz
Collecting pycopy-cpython-uerrno (from pycopy-cpython-upip)
  Downloading https://files.pythonhosted.org/packages/1d/15/984d395daf1cc8d988342ed31e9a4185c4fc840b828781a4496fc686d227/pycopy-cpython-uerrno-0.1.tar.gz
Collecting pycopy-cpython-ujson (from pycopy-cpython-upip)
  Downloading https://files.pythonhosted.org/packages/11/54/39295201aa41fc46a1ff4e1f293e9f30c54ec6591d2c56aa680785e32ffe/pycopy-cpython-ujson-0.2.tar.gz
Collecting pycopy-cpython-uos (from pycopy-cpython-upip)
  Downloading https://files.pythonhosted.org/packages/6b/3c/7acd4c35f1fba33f22b287fe40e48d8d3bce1be52daea424f1a8cc9f494b/pycopy-cpython-uos-0.1.tar.gz
Collecting pycopy-cpython-ure (from pycopy-cpython-upip)
  Downloading https://files.pythonhosted.org/packages/36/34/c00de8381f62edce97dc4ff3c38da6bcad1b56678ce97328b6fca4920b19/pycopy-cpython-ure-0.2.tar.gz
Collecting pycopy-cpython-usocket (from pycopy-cpython-upip)
  Downloading https://files.pythonhosted.org/packages/a3/02/b2f3ed686bc20174a2e99f4398dd39559a83f4cd00924acac0db0c3cafde/pycopy-cpython-usocket-0.1.tar.gz
Collecting pycopy-cpython-ussl (from pycopy-cpython-upip)
  Downloading https://files.pythonhosted.org/packages/9b/7a/cad4414aa8967fdeb5b5ed11b165a03d64669849d2c51bf5659e6ea263a3/pycopy-cpython-ussl-0.1.tar.gz
Collecting pycopy-cpython-uzlib (from pycopy-cpython-upip)
  Downloading https://files.pythonhosted.org/packages/bf/ee/c7d171d0d2fbbe9a38a3108bb8cd59db60ceaa0f7b4ddfc66199e067dd06/pycopy-cpython-uzlib-0.2.tar.gz
Collecting pycopy-cpython-micropython (from pycopy-cpython-ujson->pycopy-cpython-upip)
  Downloading https://files.pythonhosted.org/packages/6b/1a/153a82cc1d7735f8a9b81422eb13dd78f218b872e4e0142f83c2a95c2e11/pycopy-cpython-micropython-0.2.tar.gz
Collecting pycopy-cpython-uio (from pycopy-cpython-ussl->pycopy-cpython-upip)
  Downloading https://files.pythonhosted.org/packages/13/e5/50f0bcd756ebe7d2f4464f2c2b73fbb5f71f9283a49f3e374a20892e2756/pycopy-cpython-uio-0.2.tar.gz
Installing collected packages: pycopy-cpython-uerrno, pycopy-cpython-uio, pycopy-cpython-micropython, pycopy-cpython-ujson, pycopy-cpython-uos, pycopy-cpython-ure, pycopy-cpython-usocket, pycopy-cpython-ussl, pycopy-cpython-uzlib, pycopy-cpython-upip
  Running setup.py install for pycopy-cpython-uerrno ... done
  Running setup.py install for pycopy-cpython-uio ... done
  Running setup.py install for pycopy-cpython-micropython ... done
  Running setup.py install for pycopy-cpython-ujson ... done
  Running setup.py install for pycopy-cpython-uos ... done
  Running setup.py install for pycopy-cpython-ure ... done
  Running setup.py install for pycopy-cpython-usocket ... done
  Running setup.py install for pycopy-cpython-ussl ... done
  Running setup.py install for pycopy-cpython-uzlib ... done
  Running setup.py install for pycopy-cpython-upip ... done
Successfully installed pycopy-cpython-micropython-0.2 pycopy-cpython-uerrno-0.1 pycopy-cpython-uio-0.2 pycopy-cpython-ujson-0.2 pycopy-cpython-uos-0.1 pycopy-cpython-upip-1.3.3 pycopy-cpython-ure-0.2 pycopy-cpython-usocket-0.1 pycopy-cpython-ussl-0.1 pycopy-cpython-uzlib-0.2
pfalcon@x230:~/tmp$ 
pfalcon@x230:~/tmp$ 
pfalcon@x230:~/tmp$ python3 -m upip install -p cpy-upip pycopy-os
Installing to: cpy-upip/
Warning: pypi.org SSL certificate is not validated
Installing pycopy-os from https://files.pythonhosted.org/packages/4d/c3/40e63ca7cf4197da68e5d9e9ba2aedd94408117f6f0a9c0d48e9f096848f/pycopy-os-0.8.2.tar.gz
Installing pycopy-ffilib from https://files.pythonhosted.org/packages/1c/3c/764cff40c525c929e8d98e33ecd2be78218d670571c93b006b5a7f118000/pycopy-ffilib-0.1.3.tar.gz
Installing pycopy-errno from https://files.pythonhosted.org/packages/ab/7c/a0d03cb169f37dc9a9de9fd40376d6f8a781b29768c45cb22158edb11985/pycopy-errno-0.1.4.tar.gz
Installing pycopy-stat from https://files.pythonhosted.org/packages/65/95/ff904fce737681df1e6bda82c7618f1212a285b9650c73547af5757440cf/pycopy-stat-0.5.1.tar.gz
Installing pycopy-os.path from https://files.pythonhosted.org/packages/a7/9c/7bdecbad2b072604aae93ea3b2c17b962adf5693dbb60282af600c3f757b/pycopy-os.path-0.2.tar.gz
pfalcon@x230:~/tmp$ 

[amotl]

Dear Paul,

thanks a bunch for the quick fix. We can confirm the installation of pycopy-cpython-upip==1.3.3 now works without a glitch. However, when trying to install a MicroPython package as outlined above, the upip module croaks like

$ python3 -m upip install -p dist-packages pycopy-os
Installing to: dist-packages/
Warning: pypi.org SSL certificate is not validated
Installing pycopy-os from https://files.pythonhosted.org/packages/4d/c3/40e63ca7cf4197da68e5d9e9ba2aedd94408117f6f0a9c0d48e9f096848f/pycopy-os-0.8.2.tar.gz

Traceback (most recent call last):
  File "/Users/amo/dev/hiveeyes/sources/hiveeyes-micropython-firmware/.venv3/lib/python3.7/site-packages/upip.py", line 236, in install
    meta = install_pkg(pkg_spec, install_path)
  File "/Users/amo/dev/hiveeyes/sources/hiveeyes-micropython-firmware/.venv3/lib/python3.7/site-packages/upip.py", line 209, in install_pkg
    f2 = uzlib.DecompIO(f1, gzdict_sz, gzdict_buf)
TypeError: __init__() takes 3 positional arguments but 4 were given

With kind regards, Andreas.

P.S.: Feel free to advise me to open another issue for this guy if you like to keep things separated.

[pfalcon]

Make sure that the latest version of https://pypi.org/project/pycopy-cpython-uzlib/ is installed, which is 0.2. CPython's pip has some caching and not immediately obvious upgrade logic, that's why I use pip3 install --no-cache -U --user pycopy-cpython-upip command above, with it everything worked for me.

[pfalcon]

Here's it specifically:

pip3 install --help
...
  -U, --upgrade               Upgrade all specified packages to the newest available version. The handling of dependencies depends
                              on the upgrade-strategy used.
  --upgrade-strategy <upgrade_strategy>
                              Determines how dependency upgrading should be handled. "eager" - dependencies are upgraded regardless
                              of whether the currently installed version satisfies the requirements of the upgraded package(s).
                              "only-if-needed" -  are upgraded only when they do not satisfy the requirements of the upgraded
                              package(s).

So, you may need to use --upgrade-strategy eager, as it's not clear which is the default.

[pfalcon]

Or perhaps -I, --ignore-installed Ignore the installed packages (reinstalling instead). might help. All this stuff needs testing in various real-world conditions...

[amotl]

Thanks a bunch for your suggestions and support.

When using

pip install --no-cache --upgrade --upgrade-strategy eager "pycopy-cpython-upip==1.3.3"

the process later says

  Found existing installation: pycopy-cpython-uzlib 0.1
    Uninstalling pycopy-cpython-uzlib-0.1:
      Successfully uninstalled pycopy-cpython-uzlib-0.1
Successfully installed pycopy-cpython-upip-1.3.3 pycopy-cpython-uzlib-0.2

and installing MicroPython packages works without any issue then.

Using --no-cache did not help and --user would not work as we are installing from inside a virtualenv already. So, adding --upgrade --upgrade-strategy eager hits the sweet spot for us.

Thanks again for your amazing work on MicroPython and keep up the spirit.