Why export from `poetry` to requirements.txt?

[adamghill]

Why do you export from poetry to requirements.txt at https://github.com/Tobi-De/fuzzy-couscous/blob/5326d45ca85ec25a51e604d5c38944e541556f18/project_name/deploy/Dockerfile#L9? Is it faster to install via pip in the second stage?

[Tobi-De]

Ok @adamghill, when I talked about multi-stage docker I phrased my sentence wrong, I gave the impression that it was what had improved the installation speed of my requirements but it is not the case. In reality it's the layering that affected the speed of my requirements installation, I said pandas was the bottleneck right? Let me show you what I actually did (look for a comment in the snippet, you don't have to read the whole thing)

FROM python:3.11.0-bullseye as requirements-stage

WORKDIR /tmp

RUN pip install poetry

COPY ./pyproject.toml ./poetry.lock* /tmp/

RUN poetry export -f requirements.txt --output requirements.txt --without-hashes

FROM python:3.11.0-alpine3.16

WORKDIR /code

RUN apk update \
    && apk add --virtual build-deps build-base \
    && apk add --no-cache libffi-dev libpq-dev \
    && pip install --upgrade pip \
    && python --version

# THIS IS THE IMPORTANT LINE, Installing numpy, scipy
RUN pip install --no-cache-dir \
    pandas==1.5.1 \
    numpy==1.23.4

COPY --from=requirements-stage /tmp/requirements.txt /code/requirements.txt

RUN pip install --no-cache-dir --upgrade -r /code/requirements.txt

COPY . /code

EXPOSE 80

CMD ["sh", "/code/runserver.sh"]

I had identified that it was pandas and its numpy dependency that was taking time to install and build, so I installed them in a separate layer so that at the stage where all the requirements need to be installed wheels for these two packages are already available. I don't know if that was clear enough, read this for a bit more context and explanation.

Now, to answer your question, I use multi-stage mainly to save space, to have a lighter build, that's why the second stage use an alpine version and since poetry and all it dependencies are not needed to run my application, I just don't install it.

When using Poetry, it would make sense to use Docker multi-stage builds because you don't really need to have Poetry and its dependencies installed in the final container image, you only need to have the generated requirements.txt file to install your project dependencies.

The above paragraph is from this excellent guide on docker by tiangolo, I suggest you read it, it's a much better and complete explanation than anything I could write.

And finally, when you are done writing your Dockerfile, I would love to take a look at it, maybe it will inspire some changes in mine. Thanks

[adamghill]

This is the Dockerfile I landed on: https://github.com/adamghill/django-unicorn.com/blob/main/Dockerfile. It is a mix of https://github.com/michaeloliverx/python-poetry-docker-example/blob/master/docker/Dockerfile and your Dockerfile. This deploys super quick which I think is because of the multi-stage setup. I'm still an amateur at this, though, so I might be wrong!