pip.get_installed_distributions not supported by PIP

[daliu]

I'm trying to install xbos with pip 18.0, but this issue is due to outdated Docker version usage.

Collecting docker==2.5.1 (from xbos)
  Using cached https://files.pythonhosted.org/packages/ea/80/f32e70f338055922eb996b05ee41b14acde925216f4e7bba16788a377763/docker-2.5.1.tar.gz
    Complete output from command python setup.py egg_info:
    Traceback (most recent call last):
      File "<string>", line 1, in <module>
      File "/private/var/folders/ds/20h1hvs103lgv0ywgww3_ktw0000gq/T/pip-install-hooX6l/docker/setup.py", line 12, in <module>
        if 'docker-py' in [x.project_name for x in pip.get_installed_distributions()]:
    AttributeError: 'module' object has no attribute 'get_installed_distributions'

According to some other sources, this issue arises due to improper pip-module usage. https://github.com/pypa/pip/issues/5243 https://github.com/ARMmbed/mbed-cli/issues/657

Later versions of Docker have fixed this issue. https://github.com/docker/docker-py/issues/2001

Could we update the XBOS setup.py to use a later version of Docker? This will allow development on future versions of pip, without the need for continuous backward-compatible workarounds. I'm using MacPorts, which do not support downgrading the pip version.**

[gtfierro]

Thanks for digging into it to figure out what the issue is! Can you check that https://github.com/SoftwareDefinedBuildings/XBOS/pull/71 fixes the issue for you?

[daliu]

By running python setup.py sdist from source distribution, (and after I downloaded several dependency packages) I was indeed able to install xbos. However, I would like to note that I had issues installing the package by using pip install -e . from the source file.

Command "/Users/davidliu/.virtualenvs/default/bin/python -c "import setuptools, tokenize;file='/Users/davidliu/code/XBOS/python/setup.py';f=getattr(tokenize, 'open', open)(file);code=f.read().replace('\r\n', '\n');f.close();exec(compile(code, file, 'exec'))" develop --no-deps --install-purelib=/Users/davidliu/.virtualenvs/default/lib/python2.7/site-packages" failed with error code 1 in /Users/davidliu/code/XBOS/python/

Upon further expection, the issue seems to be with the "develop" keyword/variable, which I don't understand its purpose. Neither the standard Execute or Complie function have any reference to this, and I had trouble finding anything about it in PIP documentation as well. Maybe you have some idea?

Anyways, one last note. At one point, while using pip install -e . I got this issue:

#include "capnp/common.h"
         ^~~~~~~~~~~~~~~~
1 error generated.
*WARNING* no libcapnp detected or rebuild forced. Will download and build it from source now. If you have C++ Cap'n Proto installed, it may be out of date or is not being detected. Downloading and building libcapnp may take a while.
fetching https://capnproto.org/capnproto-c++-0.6.1.tar.gz into /private/var/folders/ds/20h1hvs103lgv0ywgww3_ktw0000gq/T/pip-install-B1cy0S/pycapnp/bundled
error: <urlopen error [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed (_ssl.c:726)>

----------------------------------------

Command "/Users/davidliu/.virtualenvs/default/bin/python -u -c "import setuptools, tokenize;file='/private/var/folders/ds/20h1hvs103lgv0ywgww3_ktw0000gq/T/pip-install-B1cy0S/pycapnp/setup.py';f=getattr(tokenize, 'open', open)(file);code=f.read().replace('\r\n', '\n');f.close();exec(compile(code, file, 'exec'))" install --record /private/var/folders/ds/20h1hvs103lgv0ywgww3_ktw0000gq/T/pip-record-tiHi6H/install-record.txt --single-version-externally-managed --compile --install-headers /Users/davidliu/.virtualenvs/default/bin/../include/site/python2.7/pycapnp --install-purelib=/Users/davidliu/.virtualenvs/default/lib/python2.7/site-packages" failed with error code 1 in /private/var/folders/ds/20h1hvs103lgv0ywgww3_ktw0000gq/T/pip-install-B1cy0S/pycapnp/

I wasn't sure what was using the capnproto and pycapnp packages, but installing C++11 and using it instead of the standard OSX C++ allowed the installation to work correctly. Just a heads up, in case there are future issues here.

[daliu]

At first I tried installing from source with pip install -e . which did not work.

#include "capnp/common.h"
         ^~~~~~~~~~~~~~~~
1 error generated.
*WARNING* no libcapnp detected or rebuild forced. Will download and build it from source now. If you have C++ Cap'n Proto installed, it may be out of date or is not being detected. Downloading and building libcapnp may take a while.
fetching https://capnproto.org/capnproto-c++-0.6.1.tar.gz into /private/var/folders/ds/20h1hvs103lgv0ywgww3_ktw0000gq/T/pip-install-B1cy0S/pycapnp/bundled
error: <urlopen error [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed (_ssl.c:726)>

----------------------------------------

Command "/Users/davidliu/.virtualenvs/default/bin/python -u -c "import setuptools, tokenize;file='/private/var/folders/ds/20h1hvs103lgv0ywgww3_ktw0000gq/T/pip-install-B1cy0S/pycapnp/setup.py';f=getattr(tokenize, 'open', open)(file);code=f.read().replace('\r\n', '\n');f.close();exec(compile(code, file, 'exec'))" install --record /private/var/folders/ds/20h1hvs103lgv0ywgww3_ktw0000gq/T/pip-record-tiHi6H/install-record.txt --single-version-externally-managed --compile --install-headers /Users/davidliu/.virtualenvs/default/bin/../include/site/python2.7/pycapnp --install-purelib=/Users/davidliu/.virtualenvs/default/lib/python2.7/site-packages" failed with error code 1 in /private/var/folders/ds/20h1hvs103lgv0ywgww3_ktw0000gq/T/pip-install-B1cy0S/pycapnp/

This issue was solved by installing C++11 because OSX defaults to standard C++

curl -O https://capnproto.org/capnproto-c++-0.6.1.tar.gz
tar zxf capnproto-c++-0.6.1.tar.gz
cd capnproto-c++-0.6.1
./configure
make -j6 check
sudo make install

After upgrading C++, there was another issue.

Installing collected packages: xbos, six
Running setup.py develop for xbos
Complete output from command /Users/davidliu/.virtualenvs/default/bin/python -c "import setuptools, tokenize;__file__='/Users/davidliu/code/XBOS/python/setup.py';f=getattr(tokenize, 'open', open)(__file__);code=f.read().replace('\r\n', '\n');f.close();exec(compile(code, __file__, 'exec'))" develop --no-deps --install-purelib=/Users/davidliu/.virtualenvs/default/lib/python2.7/site-packages:
['xbos', 'xbos.util', 'xbos.services', 'xbos.devices']
usage: -c [global_opts] cmd1 [cmd1_opts] [cmd2 [cmd2_opts] ...]
   or: -c --help [cmd1 cmd2 ...]
   or: -c --help-commands
   or: -c cmd --help

error: option --install-purelib not recognized

error: option --install-purelib not recognized

----------------------------------------

Command "/Users/davidliu/.virtualenvs/default/bin/python -c "import setuptools, tokenize;file='/Users/davidliu/code/XBOS/python/setup.py';f=getattr(tokenize, 'open', open)(file);code=f.read().replace('\r\n', '\n');f.close();exec(compile(code, file, 'exec'))" develop --no-deps --install-purelib=/Users/davidliu/.virtualenvs/default/lib/python2.7/site-packages" failed with error code 1 in /Users/davidliu/code/XBOS/python/

This new issue seemed to arise from:

develop --no-deps --install-purelib=/Users/davidliu/.virtualenvs/default/lib/python2.7/site-packages" failed with error code 1 in /Users/davidliu/code/XBOS/python/

I could not find the develop keyword under the standard python Exec or Compile function, and had no luck finding any reference in PIP documentation. Maybe you know what's going on here?

Anyways, I finally installed by using python setup.py sdist. I can now successfully import xbos in python 2.7.15

[daliu]

Oh, I was mistaken. Installing with python setup.py sdist seemed like it was working, but once I move out of the directory containing the xbos package, importing doesn't work anymore.

I solved this by using: python setup.py install --user

[gtfierro]

Yes, pycapnp is required for this package to work. Unfortunately, they've noted some installation issues for OS X (https://pypi.org/project/pycapnp/). I would recommend using virtualenv or pipenv to manage installation environments for your Python packages, rather than installing them to a global environment (either system or user)