pip 10 no longer provides internal API get_installed_distributions (`ImportError: cannot import name get_installed_distributions`)

[dmr-x]

A user on #tor of irc.oftc.net reported this problem after installing theonionbox:

(theonionbox) ?  theonionbox theonionbox
        23:13:40.049 The Onion Box: WebInterface to monitor Tor node operations.
        23:13:40.053 Version v4.1.3 (stamp 20180212|224648)
        23:13:40.053 Running on a Linux host.
        23:13:40.053 Running with permissions of user 'user'.
        23:13:40.054 Python version is 2.7.13
                     (/home/user/theonionbox/bin/python).
        23:13:40.055 This seems to be a Python VirtualEnv.
        23:13:40.067 No (valid) configuration file found; operating with default
                     settings.
Traceback (most recent call last):
  File "/home/user/theonionbox/bin/theonionbox", line 11, in <module>
    sys.exit(main())
  File "/home/user/theonionbox/local/lib/python2.7/site-packages/theonionbox/__main__.py", line 18, in main
    from .theonionbox import main as onion_main
  File "/home/user/theonionbox/local/lib/python2.7/site-packages/theonionbox/theonionbox.py", line 648, in <module>
    from pip import get_installed_distributions
ImportError: cannot import name get_installed_distributions

I reproduced the import problem in an interactive python2 (2.7.13) session locally, in a fresh virtualenv:

>>> from pip import get_installed_distributions
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
ImportError: cannot import name get_installed_distributions

It is related to pip 10. I confirmed that the import works in a pip 9 virtualenv I had around.

My environment, for reference:

$ pip freeze --all
pip==10.0.0
pkg-resources==0.0.0
setuptools==39.0.1
wheel==0.31.0

I searched to root cause this to pip 10.

Please see this pip issue for context: pypa/pip#5154

And see this thread on the Distutils-SIG mailing list for additional context.

Notably, see this issue comment which indicates:

The use of pip as a module was never been supported, and the internal APIs have been reorganised in pip 10.

I haven't looked at theonionbox's code, but this message on the mailing list might provide a solution.

theonionbox will need to change in order to accommodate this. Right now it appears that every user of theonionbox in a new virtualenv will encounter this issue unless they somehow get an older version of pip.

[ralphwetzel]

Good Morning. Unfortunately I have to confirm that this change in pip10 broke a part of the startup verification procedure of TOB. I‘m going to provide an initial fix later today; it yet might need some days to arrive in pypi. BR, Ralph

[ralphwetzel]

As initial action I've altered theonionbox.py to swallow the exception that is raised on trying to import get_installed_distribution with pip10. The relevant fix is in Lines 647 - 669 - precisely in 648 & 668/669. You can either grab this file from GitHub to replace the version currently installed on your system - or perform this minor mod by hand.

A sustainable fix will be provided with the next release.

[Ricks-ha]

Even with the line edited it is failing providing the following issue.

Traceback (most recent call last): File "/home/ricks/theonionbox/bin/theonionbox", line 11, in <module> sys.exit(main()) File "/home/ricks/theonionbox/local/lib/python2.7/site-packages/theonionbox/__main__.py", line 18, in main from .theonionbox import main as onion_main File "/home/ricks/theonionbox/local/lib/python2.7/site-packages/theonionbox/theonionbox.py", line 652 boxLog.debug('Required packages version verification:') ^ SyntaxError: invalid syntax So the fix basically does not fix the problem.

[ralphwetzel]

The fix fixed this issue - yet induced a SyntaxError. Please make sure to indent the block covered by the try: ... except: ... guards as shown below:

https://github.com/ralphwetzel/theonionbox/blob/1720612fad53602674b909be2c35364b4acbfcd9/theonionbox/theonionbox.py#L647-L669

This was not committed correctly in the last version of theonionbox.py. Fixed. Thank you!

[Ricks-ha]

Thanks for fixing!

[ralphwetzel]

Finally fixed with the release of The Onion Box v4.2. Closed.