Conflicting with version dependencies when running pip install

[pmventura]

Hi everyone,

Having issues with version dependencies when running pip install on docker. However, when installing on my mac without docker and just virtualenv, works perfectly fine.

Mac OS - macOS Mojave v10.14. Python Version - v3.7.3 Docker Compose Version - version 1.27.4, build 40524192

Here's the first error I got when running the docker-compose up

ERROR: Cannot install -r ./python-project/requirements/base.txt (line 104), -r ./python-project/requirements/base.txt (line 109), -r ./python-project/requirements/base.txt (line 114), -r ./python-project/requirements/base.txt (line 116), -r ./python-project/requirements/base.txt (line 141), -r ./python-project/requirements/base.txt (line 144), -r ./python-project/requirements/base.txt (line 145), -r ./python-project/requirements/base.txt (line 87) and six==1.10.0 because these package versions have conflicting dependencies.

The conflict is caused by:
    The user requested six==1.10.0
    bcrypt 3.2.0 depends on six>=1.4.1
    cvxpy 1.0.25 depends on six
    django-anymail 0.5 depends on six
    django-compat 1.0.15 depends on six>=1.10.0
    django-extensions 2.2.1 depends on six>=1.2
    drf-yasg 1.16.1 depends on six>=1.10.0
    fake-factory 0.6.0 depends on six
    google-api-core 1.26.0 depends on six>=1.13.0

To fix this you could try to:
1. loosen the range of package versions you've specified
2. remove package versions to allow pip attempt to solve the dependency conflict

ERROR: ResolutionImpossible: for help visit https://pip.pypa.io/en/latest/user_guide/#fixing-conflicting-dependencies
ERROR: Service 'backend' failed to build : The command '/bin/sh -c pip install -r ./python-project/requirements/base.txt' returned a non-zero code: 1

I tried to loosen the six package from six==1.10.0 to six>=1.2,<=1.15.0

And throws a different error for requests package this time. Here's the error

ERROR: Cannot install -r ./python-project/requirements/base.txt (line 1), -r ./python-project/requirements/base.txt (line 10), -r ./python-project/requirements/base.txt (line 109), -r ./python-project/requirements/base.txt (line 13), -r ./python-project/requirements/base.txt (line 145), -r ./python-project/requirements/base.txt (line 159), -r ./python-project/requirements/base.txt (line 166), -r ./python-project/requirements/base.txt (line 79), -r ./python-project/requirements/base.txt (line 81), -r ./python-project/requirements/base.txt (line 83), -r ./python-project/requirements/base.txt (line 9), -r ./python-project/requirements/base.txt (line 97) and requests==2.22.0 because these package versions have conflicting dependencies.

The conflict is caused by:
    The user requested requests==2.22.0
    adal 1.2.6 depends on requests<3 and >=2.0.0
    azure-cosmosdb-table 1.0.6 depends on requests
    azure-datalake-store 0.0.51 depends on requests>=2.20.0
    azure-keyvault 1.1.0 depends on requests>=2.18.4
    azure-servicebus 0.21.1 depends on requests
    azure-servicemanagement-legacy 0.20.7 depends on requests
    azure-storage-common 1.4.2 depends on requests
    coreapi 2.3.3 depends on requests
    django-anymail 0.5 depends on requests>=2.4.3
    google-api-core 1.26.0 depends on requests<3.0.0dev and >=2.18.0
    jira 2.0.0 depends on requests>=2.10.0
    microsofthealth 0.1.0 depends on requests==2.12.1

To fix this you could try to:
1. loosen the range of package versions you've specified
2. remove package versions to allow pip attempt to solve the dependency conflict

ERROR: ResolutionImpossible: for help visit https://pip.pypa.io/en/latest/user_guide/#fixing-conflicting-dependencies
ERROR: Service 'backend' failed to build : The command '/bin/sh -c pip install -r ./python-project/requirements/base.txt' returned a non-zero code: 1

After changing the package from requests==2.22.0 to requests>=2.0.0,<=2.25.1. Now, it takes ages to finish the installation.

Collecting azure-common~=1.1
  Downloading azure_common-1.1.25-py2.py3-none-any.whl (12 kB)
  Downloading azure_common-1.1.24-py2.py3-none-any.whl (12 kB)
  Downloading azure_common-1.1.23-py2.py3-none-any.whl (11 kB)
  Downloading azure_common-1.1.22-py2.py3-none-any.whl (11 kB)
  Downloading azure_common-1.1.21-py2.py3-none-any.whl (11 kB)
  Downloading azure_common-1.1.20-py2.py3-none-any.whl (11 kB)
  Downloading azure_common-1.1.19-py2.py3-none-any.whl (11 kB)
  Downloading azure_common-1.1.18-py2.py3-none-any.whl (12 kB)
  Downloading azure_common-1.1.17-py2.py3-none-any.whl (10 kB)
  Downloading azure_common-1.1.16-py2.py3-none-any.whl (10 kB)
  Downloading azure_common-1.1.15-py2.py3-none-any.whl (10 kB)
  Downloading azure_common-1.1.14-py2.py3-none-any.whl (10 kB)
  Downloading azure_common-1.1.13-py2.py3-none-any.whl (10 kB)
  Downloading azure_common-1.1.12-py2.py3-none-any.whl (9.7 kB)
  Downloading azure_common-1.1.11-py2.py3-none-any.whl (9.7 kB)
  Downloading azure_common-1.1.10-py2.py3-none-any.whl (9.6 kB)
  Downloading azure_common-1.1.9-py2.py3-none-any.whl (10 kB)
  Downloading azure_common-1.1.8-py2.py3-none-any.whl (7.9 kB)
  Downloading azure_common-1.1.7-py2.py3-none-any.whl (7.7 kB)
  Downloading azure_common-1.1.6-py2.py3-none-any.whl (5.9 kB)
  Downloading azure_common-1.1.5-py2.py3-none-any.whl (4.1 kB)
  Downloading azure_common-1.1.4-py2.py3-none-any.whl (4.9 kB)
  Downloading azure_common-1.1.3-py2.py3-none-any.whl (4.9 kB)
  Downloading azure_common-1.1.2-py2.py3-none-any.whl (4.9 kB)
  Downloading azure_common-1.1.1-py2.py3-none-any.whl (4.8 kB)
  Downloading azure_common-1.1.0-py2.py3-none-any.whl (4.8 kB)
INFO: pip is looking at multiple versions of azure-cosmosdb-table to determine which version is compatible with other requirements. This could take a while.
INFO: This is taking longer than usual. You might need to provide the dependency resolver with stricter constraints to reduce runtime. If you want to abort this run, you can press Ctrl + C to do so. To improve how pip performs, tell us what happened here: https://pip.pypa.io/surveys/backtracking
INFO: pip is looking at multiple versions of azure-common to determine which version is compatible with other requirements. This could take a while

[pmventura]

For docker-compose.yml

FROM python:3.7
ENV PYTHONUNBUFFERED 1
ENV TERM xterm

RUN echo "deb http://ftp.de.debian.org/debian buster main" > /etc/apt/sources.list.d/buster.list

RUN apt-get update -y &&\
    apt-get install -y gfortran libopenblas-dev liblapack-dev git cron supervisor vim less libmagic1 poppler-utils build-essential python3-cffi libcairo2 libpango-1.0-0 libpangocairo-1.0-0 libgdk-pixbuf2.0-0 libffi-dev shared-mime-info tesseract-ocr fonts-roboto fonts-lato fonts-dancingscript &&\
    apt-get clean &&\
    rm -rf /var/lib/apt/lists/*

# upgrade pip
RUN pip install --upgrade pip setuptools==41.2.0 requests==2.22.0

RUN pip install numpy==1.17.1
RUN pip install pandas==0.25.1
RUN pip install scipy==1.3.1
ADD requirements/base.txt ./python-project/requirements/base.txt
RUN pip install -r ./python-project/requirements/base.txt

ADD . ./python-project

EXPOSE 80

RUN ln -s /python-project/devop/supervisor-app.conf /etc/supervisor/conf.d/
COPY ./docker-entrypoint.sh /
COPY /local_settings_docker.py /python-project/local_settings.py
RUN chmod +x /docker-entrypoint.sh

ENTRYPOINT ["/docker-entrypoint.sh"]
CMD ["backend"]

[pmventura]

For base.txt or known as (requirements.txt)

django-filebrowser==3.10.1
django-tinymce==2.7.0
Django==2.2.7

numpy==1.17.1
pandas==0.25.1
pandas-datareader==0.8.1
CVXcanon==0.0.23.3
cvxopt==1.2.3
cvxpy==1.0.25
scikit-learn==0.21.3
scipy==1.3.1
statsmodels==0.10.1
django-pandas==0.6.0
xlrd==1.2.0

django-redis==4.10.0

requests==2.22.0
beautifulsoup4==4.5.1
lxml==4.4.1

diff-match-patch==20121119
django-anymail==0.5
django-authority==0.9
django-ckeditor==5.6.1
django-common-helpers==0.7.0
django-compat==1.0.15
django-cron==0.5.1
celery==4.2.1

django-filter==2.2.0
django-flat-theme==1.1.3
django-haystack==2.4.1
django-import-export==1.0.1
django-localflavor-au-tjh==1.1.2
django-mptt==0.7.4
django_nose==1.4.5
django-nested-admin==3.1.3

django-phonenumber-field==1.1.0
-e git://github.com/darklow/django-suit.git@v2#egg=django-suit
django-taggit==0.17.5
git+git://github.com/chibisov/drf-extensions@33b50a8d2ba21bf1a612333ea5d5aea845534eb2
ecos==2.0.4
Markdown==2.6.5
pexpect==4.0.1
Pillow==5.2.0
pydenticon==0.2
pinax-eventlog==2.0.3
psycopg2-binary==2.7.5
ptyprocess==0.5
PyPDF2==1.26.0
pysftp==0.2.8
python-dateutil==2.8.0
pytz==2018.7
tzlocal==1.3
scs==1.1.7
six==1.10.0
git+git://github.com/kennethreitz/tablib@ddc4bd30f22e40ae4e3aa9347747f1f96912191a
toolz==0.10.0
ujson==1.34
sqlparse==0.3.0
cairocffi==1.1.0
cffi==1.13
CairoSVG==2.4.2
pbr==5.1.1
weasyprint==50

django-test-without-migrations==0.6

django-bootstrap3==11.0.0

djangorestframework==3.9
drf-yasg==1.16.1
packaging==19.1
plaid-python==2.3.3
google-api-python-client>=1.6.2
MicrosoftHealth>=0.1.0
git+https://github.com/maximebf/python-withings.git
stripe==1.53.0
oauth2client==4.1.2

django-model-utils==2.3.1
# re-enable after fixing # django-notifications-hq>=1.0 # upgrade to 1.0 later
django-extensions==2.2.1 # to visualize models

azure==4.0.0
django-storages==1.7.1

git+git://github.com/blampe/IbPy.git

django-recurrence==1.3.0

pygeoip==0.3.2

jira==2.0.0

python-gnupg==0.4.0

[yosifkit]

Honestly this sounds like your requirements.txt is not specific enough to reproduce the correct set of installed packages. I don't see this as an issue with the Docker image. I would guess that running with venv it has already set up your local directory and so already has the set of packages installed and doesn't need to do any dependency satisfiability solving.

In the future, it'd be better to post questions like this in the Docker Community Forums, the Docker Community Slack, or Stack Overflow.

[dswah]

@pmventura Did you solve this? i'm seeing similar issues, and it appears to be due to a problem with the dependency resolver introduced in pip 21.2.