bentoml / BentoML

The easiest way to serve AI apps and models - Build Model Inference APIs, Job queues, LLM apps, Multi-model pipelines, and more!
https://bentoml.com
Apache License 2.0
7.16k stars 792 forks source link

bug: pip not found when using cuda based image with my own Dockerfile template #4900

Closed shanirosen-airis closed 3 months ago

shanirosen-airis commented 3 months ago

Describe the bug

Hi there, when creating a new bento service with cuda (any cuda version) in the new version (1.3.1), my bentoml build started to fail on: pip not found when trying to run the pip install here in my dockerfile tmeplate:

{% extends bento_base_template %}
{% block SETUP_BENTO_BASE_IMAGE %}
ARG CODEARTIFACT_USER
{{ super() }}

ARG CODEARTIFACT_USER
ENV CODEARTIFACT_USER=$CODEARTIFACT_USER
ADD src/requirements.txt .

{% endblock %}
{% block SETUP_BENTO_COMPONENTS %}
{{ super() }}

RUN --mount=type=secret,id=code_artifact_token \
    export CODEARTIFACT_TOKEN=$(cat /run/secrets/code_artifact_token) && \
    pip config set global.extra-index-url https://${CODEARTIFACT_USER}:${CODEARTIFACT_TOKEN}@######.d.codeartifact.us-east-1.amazonaws.com/pypi/######/simple/ && \
    pip install -r src/requirements.txt

{% endblock %}

When comparing to 1.2.20 version I noticed these lines:

RUN curl -O https://bootstrap.pypa.io/get-pip.py && \
    python3 get-pip.py && \
    rm -rf get-pip.py

were missing from the cuda template file (for example bentoml/_internal/container/frontend/dockerfile/templates/cuda_debian.j2)

These lines existed in the templates under 1.2.20 and in that version I had no problems running pip install through my own template.

example bentofile.yaml:

service: "service:myservice"
description: |
  ## This is a sample module
include:
  - "src/*"
  - "*.py"
  - "requirements.txt"
exclude:
  - "tests/*"
  - "*venv/*"
docker:
  distro: debian
  dockerfile_template: ./Dockerfile.template.jinja
  cuda_version: "12.1.1"

I don't mind opening a pr for that fix if you this that is indeed the problem :)

To reproduce

  1. create a simple bentofile with a simple project like the bentofile.yaml above with a cuda version
  2. create a Dockerfile template jinja file like the provided above (can do anything involving pip, even just RUN pip --version)
  3. run bentoml build and see the error

Expected behavior

In 1.2.20 the build is perfectly fine and the pip install works without any problems.

Environment

bentoml: 1.3.1 python: 3.10.9 platform: Linux-5.15.0-1051-aws-x86_64-with-glibc2.31

frostming commented 3 months ago

We have switched to uv as the package installer, please either use uv or install pip in the template yourself.

shanirosen-airis commented 3 months ago

alright, good to know, thanks!