Install hangs when installing local package from root directory

brynpickering commented 12 months ago

Description

We have a Dockerfile for our project which has successfully built Docker images of the package until a change we made away from setup.py to pyproject.toml 7 days ago.

This issue occurs on an attempt locally to build the image and in AWS CodeBuild.

Project repo: https://github.com/arup-group/genet

The pip install [options] . step gets stuck at this stage:

Step 8/8 : RUN pip3 install --no-cache-dir --compile -e . && pip cache purge
 ---> Running in 921fd92eb695
Obtaining file:///
  Installing build dependencies: started
  Installing build dependencies: finished with status 'done'
  Checking if build backend supports build_editable: started
  Checking if build backend supports build_editable: finished with status 'done'
  Getting requirements to build editable: started

Tests locally in which I change the options on the pip call lead to the same result. That is, there is no difference between pip install . and pip3 install --no-cache-dir --compile -e ..

Running this same command outside Docker works as expected (e.g., our github CI runs fine). It is only within Docker that we have the issue.

This might be better placed as an issue in https://github.com/pypa/setuptools/ or https://github.com/pypa/build. If running pip install build && python -m build, the build hangs at an equivalent place but with a slightly different log message ("Getting build dependencies for sdist").

Expected behavior

Docker image builds successfully in a matter of minutes and the Getting requirements to build editable step takes <1 second (as it does outside the Docker build).

pip version

23.3.1 (also tried with 22.x amd 23.2.x)

Python version

3.11.4 (also tried with 3.10)

OS

MacOS / Linux

How to Reproduce

Clone https://github.com/arup-group/genet
Attempt to build Docker image (docker build -t genet:latest)

Output

Step 5/8 : RUN /usr/local/bin/python -m pip install --no-cache-dir --compile --upgrade pip
---> Running in 79bc03d7ae7e
Requirement already satisfied: pip in /usr/local/lib/python3.11/site-packages (23.1.2)
Collecting pip
 Downloading pip-23.3.1-py3-none-any.whl (2.1 MB)
    ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 2.1/2.1 MB 183.8 MB/s eta 0:00:00
Installing collected packages: pip
 Attempting uninstall: pip
   Found existing installation: pip 23.1.2
   Uninstalling pip-23.1.2:
     Successfully uninstalled pip-23.1.2
Successfully installed pip-23.3.1
WARNING: Running pip as the 'root' user can result in broken permissions and conflicting behaviour with the system package manager. It is recommended to use a virtual environment instead: https://pip.pypa.io/warnings/venv
---> 6ab71c69ea81
Removing intermediate container 79bc03d7ae7e
Step 6/8 : COPY . .
---> 93f02cf9e287
Step 7/8 : RUN pip3 install cmake>=3
---> Running in bd21027f20ba
WARNING: Running pip as the 'root' user can result in broken permissions and conflicting behaviour with the system package manager. It is recommended to use a virtual environment instead: https://pip.pypa.io/warnings/venv
---> 491faa040400
Removing intermediate container bd21027f20ba
Step 8/8 : RUN pip3 install --no-cache-dir --compile -e . && pip cache purge
---> Running in 921fd92eb695
Obtaining file:///
 Installing build dependencies: started
 Installing build dependencies: finished with status 'done'
 Checking if build backend supports build_editable: started
 Checking if build backend supports build_editable: finished with status 'done'
 Getting requirements to build editable: started

Code of Conduct

[X] I agree to follow the PSF Code of Conduct.

pfmoore commented 12 months ago

This looks like a network issue. Can the container access PyPI? You might get more informative messages if you add -vvv to the pip command.

brynpickering commented 12 months ago

You can see that it downloads latest pip in the Output section, so I don't think it's a network issue. Based on tests I did yesterday, -vvv provides nothing extra for the step where it hangs. I can run it with that option to elaborate on the preceding steps but will need to do so after a system update. My Docker container runtime is not cooperating this morning...

pfmoore commented 12 months ago

Sorry, I'm not that familiar with docker. I thought "Removing intermediate container..." followed by "Running in (different ID)" meant it might be running with a different context.

Sorry, I don't have any other ideas.

brynpickering commented 12 months ago

OK, I've rerun it with all pip calls in one RUN call and using -vvv:

Input Dockerfile:

FROM python:3.11.4-bullseye

RUN apt-get update && \
apt-get upgrade -y && \
apt-get -y install gcc git libgdal-dev libgeos-dev libspatialindex-dev curl coinor-cbc build-essential cmake libboost-dev libexpat1-dev zlib1g-dev libbz2-dev &&  \
rm -rf /var/lib/apt/lists/*

COPY . .

RUN pip install --upgrade pip && pip install -vvv . && pip cache purge

Input command: docker build -t "genet:test" --progress plain .

Output (`apt-get` `RUN` call is loaded from a cached intermediate container)

``` #0 building with "default" instance using docker driver #1 [internal] load build definition from Dockerfile #1 transferring dockerfile: 428B 0.0s done #1 DONE 0.0s #2 [internal] load .dockerignore #2 transferring context: 115B 0.0s done #2 DONE 0.1s #3 [internal] load metadata for docker.io/library/python:3.11.4-bullseye #3 DONE 1.5s #4 [1/4] FROM docker.io/library/python:3.11.4-bullseye@sha256:4b3c9c338fdf1db596eb1ccf83597b879098aecf30479a9f01839ab1f1cf0772 #4 DONE 0.0s #5 [internal] load build context #5 transferring context: 498.08kB 0.4s done #5 DONE 0.4s #6 [2/4] RUN apt-get update && apt-get upgrade -y && apt-get -y install gcc git libgdal-dev libgeos-dev libspatialindex-dev curl coinor-cbc build-essential cmake libboost-dev libexpat1-dev zlib1g-dev libbz2-dev && rm -rf /var/lib/apt/lists/* #6 CACHED #7 [3/4] COPY . . #7 DONE 2.2s #8 [4/4] RUN pip install --upgrade pip && pip install -vvv . && pip cache purge #8 12.70 Requirement already satisfied: pip in /usr/local/lib/python3.11/site-packages (23.1.2) #8 13.52 Collecting pip #8 13.77 Downloading pip-23.3.1-py3-none-any.whl (2.1 MB) #8 14.13 ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 2.1/2.1 MB 6.4 MB/s eta 0:00:00 #8 14.58 Installing collected packages: pip #8 14.58 Attempting uninstall: pip #8 14.60 Found existing installation: pip 23.1.2 #8 14.95 Uninstalling pip-23.1.2: #8 16.18 Successfully uninstalled pip-23.1.2 #8 25.04 Successfully installed pip-23.3.1 #8 25.04 WARNING: Running pip as the 'root' user can result in broken permissions and conflicting behaviour with the system package manager. It is recommended to use a virtual environment instead: https://pip.pypa.io/warnings/venv #8 28.11 Using pip 23.3.1 from /usr/local/lib/python3.11/site-packages/pip (python 3.11) #8 28.12 Non-user install because site-packages writeable #8 28.27 Created temporary directory: /tmp/pip-build-tracker-ylz8m6l2 #8 28.27 Initialized build tracking at /tmp/pip-build-tracker-ylz8m6l2 #8 28.27 Created build tracker: /tmp/pip-build-tracker-ylz8m6l2 #8 28.28 Entered build tracker: /tmp/pip-build-tracker-ylz8m6l2 #8 28.28 Created temporary directory: /tmp/pip-install-q49i1l33 #8 28.29 Created temporary directory: /tmp/pip-ephem-wheel-cache-isxdj0ye #8 28.43 Processing / #8 28.43 Added file:/// to build tracker '/tmp/pip-build-tracker-ylz8m6l2' #8 28.45 Created temporary directory: /tmp/pip-build-env-501c7fc2 #8 28.46 Installing build dependencies: started #8 28.46 Running command pip subprocess to install build dependencies #8 30.54 Using pip 23.3.1 from /usr/local/lib/python3.11/site-packages/pip (python 3.11) #8 32.85 Collecting setuptools #8 32.86 Obtaining dependency information for setuptools from https://files.pythonhosted.org/packages/bb/e1/ed2dd0850446b8697ad28d118df885ad04140c64ace06c4bd559f7c8a94f/setuptools-69.0.2-py3-none-any.whl.metadata #8 33.11 Downloading setuptools-69.0.2-py3-none-any.whl.metadata (6.3 kB) #8 33.55 Downloading setuptools-69.0.2-py3-none-any.whl (819 kB) #8 33.75 ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 819.5/819.5 kB 4.8 MB/s eta 0:00:00 #8 34.57 Installing collected packages: setuptools #8 38.95 Successfully installed setuptools-69.0.2 #8 38.96 WARNING: Running pip as the 'root' user can result in broken permissions and conflicting behaviour with the system package manager. It is recommended to use a virtual environment instead: https://pip.pypa.io/warnings/venv #8 39.31 Installing build dependencies: finished with status 'done' #8 39.33 Getting requirements to build wheel: started #8 39.33 Running command Getting requirements to build wheel ```

dimbleby commented 12 months ago

Obtaining file:///

looks like the sort of thing that might confuse something. Try running in a directory other than /?

brynpickering commented 12 months ago

That's the ticket. This works:

COPY . ./src

RUN pip3 install --no-cache-dir --compile -e ./src && pip cache purge

It's relatively common to copy things across into a Docker image like this and this issue seems to only have cropped up with our use of pyproject.toml. So, even with a straightforward workaround, I would still say that it should be able to handle installing from /. Or, at least, it should be made more obvious why it is hanging (maybe just more verbose logging of whatever it is doing following the Getting requirements to build editable message?).

pypa / pip