Conflicting sub-dependencies with other libraries

[pmenegas]

Hello. First of all, I´d like to thank you guys for the awesome library ! I am having an issue while trying to use it, I´ve managed to fix it by manually searching for conflicting dependencies inside pipenv, which leads me to believe this issue may be related to undeclared dependencies withing the build-system you guys are using on the package !

Here is the issue I face to run the app after installing pyxirr:

PS C:\codebase\my_path> & C:/Users/myuser/.virtualenvs/my_path-r724wWIu/Scripts/python.exe my_path/main.py
Traceback (most recent call last):
  File "C:\Users\myuser\.virtualenvs\my_path-r724wWIu\Lib\site-packages\requests_toolbelt\_compat.py", line 48, in <module>
    from requests.packages.urllib3.contrib import appengine as gaecontrib
ImportError: cannot import name 'appengine' from 'requests.packages.urllib3.contrib' (C:\Users\myuser\.virtualenvs\my_path-r724wWIu\Lib\site-packages\urllib3\contrib\__init__.py)
    from ..util.multipart_stream import MultipartStream
  File "C:\Users\myuser\.virtualenvs\my_path-r724wWIu\Lib\site-packages\boxsdk\util\multipart_stream.py", line 3, in <module>
    from requests_toolbelt.multipart.encoder import MultipartEncoder
  File "C:\Users\myuser\.virtualenvs\my_path-r724wWIu\Lib\site-packages\requests_toolbelt\__init__.py", line 12, in <module>
    from .adapters import SSLAdapter, SourceAddressAdapter
  File "C:\Users\myuser\.virtualenvs\my_path-r724wWIu\Lib\site-packages\requests_toolbelt\adapters\__init__.py", line 12, in <module>
    from .ssl import SSLAdapter
  File "C:\Users\myuser\.virtualenvs\my_path-r724wWIu\Lib\site-packages\requests_toolbelt\adapters\ssl.py", line 16, in <module>
    from .._compat import poolmanager
  File "C:\Users\myuser\.virtualenvs\my_path-r724wWIu\Lib\site-packages\requests_toolbelt\_compat.py", line 50, in <module>
    from urllib3.contrib import appengine as gaecontrib
ImportError: cannot import name 'appengine' from 'urllib3.contrib' (C:\Users\myuser\.virtualenvs\my_path-r724wWIu\Lib\site-packages\urllib3\contrib\__init__.py)

steps to reproduce:

• with pipenv, try to install pyxirr while having a conflicting package that uses an undeclared dependency. When running the main.py script you will start having these errors due to uncompatible libs (in my case, boxsdk 1.8-2.2)

[pmenegas]

Additional info: I´ve notticed also that even removing the package that caused the dependency does not solve the issue. I have to regenerate my virtualenv using pipenv to fix it, this is another indication to me that it is due to an undeclared dependency. I´m available to provide extra info if you guys need it ! :) regenerate = pipenv --rm, then remove the conflicting package from Pipfile and regenerate the Pipfile.lock through pipenv install command.

[Anexen]

Hi @pmenegas. pyxirr has no external dependencies. You probably have issues with other libraries (or pipenv).

[Anexen]

You should probably use --keep-outdated flag when using pipenv.

[pmenegas]

Hey @Anexen , thanks for the quick reply. It is not a problem with other dependencies (or pipenv) This happened with several other colleagues trying to use your library as well. It also happens across OSs. I understand there are no external dependencies, but this may be due to the fact that the build system you guys use turns out to be imcompatible with other libraries. Believe me we have been tackling this problem for some time now and it is definitely coming from pyxirr.

[Anexen]

pyxirr uses maturin build system. In turn, the only dependence of maturin is tomli, a pure Python TOML parser with zero dependencies. But it shouldn't matter, because the build system is only used when installing from source (dist). pyxirr is distributed via binaries (wheels) that do not require a build system to be installed. pyxirr has no way to affect the dependencies of the requests.

[Anexen]

I suggest you take a closer look at requests-toolbelt. You may need to pin urllib3 to specific version.

[pmenegas]

Hello Anexen, thanks a lot for the answers ! What led me to believe the issue was related to pyxirr was the fact that inside the application I was contributing to, we use pipenv as our go to virtualenv manager. I believe the issue comes exactly from the binaries on the wheels that have conflicts between each other. So maybe you are right and we wrongly narowed it down to a dependency issue, but there is still something fishy going on. I dunno if you understand how pipenv works, so sorry if I´m preaching to the choir, but it has an internal resolution tool which uses the Pipfile.lock in order to keep track of it´s dependency resolution phases. On this particular project, we use a whole lot of dependencies, so our lockfile is pretty big. The thing that started to happen, and which led us to believe it was due to untracked dependencies, was that after installing pyxirr through pipenv, we would always get this same error as posted above, regardless if we did pipenv uninstall pyxirr. Basically after installing pyxirr, the only way for us to get our environment going again was to completely wipe ir out with pipenv --rm and then revert Pipfile and Pipfile.lock to the previous state. If you want I can help you reproduce the error so we can investigate it. It definitely seems like we might be into something worth checking into here

[Anexen]

I used pipenv a few years ago and gave it up because unacceptable dependency updates were a frequent problem. There are a lot of issues related to unexpected package updates (1, 2, 3)

I assume the following has happened:

1. You ran pipenv install pyxirr.
2. pipenv installed pyxirr and updated all dependencies to the latest allowed version (in Pipfile).
3. The latest version of urllib3 turned out to be incompatible with your requests / requests-toolbelt version.

You need to carefully examine Pipfile.lock and pin the necessary versions in the Pipfile.

if you are sure that the reason is pyxirr, please provide a reproducible example.