codalab / codabench

Codabench is a flexible, easy-to-use and reproducible benchmarking platform. Check our paper at Patterns Cell Press https://hubs.li/Q01fwRWB0
Apache License 2.0
76 stars 28 forks source link

Poetry #1537

Closed ihsaan-ullah closed 4 months ago

ihsaan-ullah commented 4 months ago

@Didayolo, @bbearce

Question 1

In the project we have these two files:

When I want to update a package, I should update pyproject.toml right?

Question 2

In the Dockerfile the specified python version is 3.9 but in pyproject.toml it is ^3.10. Is this inconsistent or on purpose?

Question 3

In the Dockerfile we have these lines:

COPY pyproject.toml ./
COPY poetry.lock ./

RUN poetry install

This means we are copying pyproject.toml and poetry.lock. Will RUN poetry install regenerate the lock file?


I am asking these because I am updating a package bpython in pyproject.toml and I am rebuilding the django container which gives me this error:

[+] Building 2.9s (13/14)                                                                                                                                                                
 => [internal] load build definition from Dockerfile                                                                                                                                0.0s
 => => transferring dockerfile: 32B                                                                                                                                                 0.0s
 => [internal] load .dockerignore                                                                                                                                                   0.0s
 => => transferring context: 34B                                                                                                                                                    0.0s
 => [internal] load metadata for docker.io/library/python:3.9                                                                                                                       2.1s
 => [auth] library/python:pull token for registry-1.docker.io                                                                                                                       0.0s
 => [1/9] FROM docker.io/library/python:3.9@sha256:47d6f16aa0de11f2748c73e7af8d40eaf44146c6dc059b1d0aa1f917f8c5cc58                                                                 0.0s
 => [internal] load build context                                                                                                                                                   0.0s
 => => transferring context: 1.78kB                                                                                                                                                 0.0s
 => CACHED [2/9] RUN apt-get update && apt-get install -y gcc build-essential && rm -rf /var/lib/apt/lists/*                                                                        0.0s
 => CACHED [3/9] RUN curl -sSL https://install.python-poetry.org | python3 -                                                                                                        0.0s
 => CACHED [4/9] RUN poetry config virtualenvs.create false                                                                                                                         0.0s
 => CACHED [5/9] RUN poetry config virtualenvs.in-project false                                                                                                                     0.0s
 => [6/9] COPY pyproject.toml ./                                                                                                                                                    0.0s
 => [7/9] COPY poetry.lock ./                                                                                                                                                       0.0s
 => ERROR [8/9] RUN poetry install                                                                                                                                                  0.7s
------                                                                                                                                                                                   
 > [8/9] RUN poetry install:                                                                                                                                                             
#0 0.383 The currently activated Python version 3.9.19 is not supported by the project (^3.10).                                                                                          
#0 0.383 Trying to find and use a compatible version.                                                                                                                                    
#0 0.420 Using python3.11 (3.11.2)                                                                                                                                                       
#0 0.421 Skipping virtualenv creation, as specified in config file.
#0 0.490 Installing dependencies from lock file
#0 0.634 
#0 0.634 pyproject.toml changed significantly since poetry.lock was last generated. Run `poetry lock [--no-update]` to fix the lock file.
------
failed to solve: executor failed running [/bin/sh -c poetry install]: exit code: 1
(base) ihsanullah@Pixel-io codabench % 
Didayolo commented 4 months ago

For the question I'll let @bbearce answer.

I am asking these because I am updating a package bpython in pyproject.toml and I am rebuilding the django container which gives me this error

Are you using the updated version of the Dockerfile.flower? #1520

ihsaan-ullah commented 4 months ago

Are you using the updated version of the Dockerfile.flower? #1520

No, but I have already built the flower from the dockerfile in #1520. I am just trying to build django by deleting the image and then running docker-compose up -d

bbearce commented 4 months ago

Question 1: "When I want to update a package, I should update pyproject.toml right?"

Yes for both. We need to edit the toml file and then we need to recreate the lock file. It is a little complex but here is a quick way to do it:

When services are running, make change to pyproject.toml, as it is mounted in with -v flag. Then regenerate lock with:

docker compose exec django poetry lock

because it is mounted in, it should be available in the codabench directory.

It gets a little more complex if you change the python version of Dockerfile. In that case, build docker image with new python and use new image with above command.

Question 2: Inconsistent and should be 3.9 or something more speccific.

Question 3: No, poetry install will install the lock packages exactly as they are. To regenerate lock file we need poetry lock.