search

bentoml / BentoML

The easiest way to serve AI apps and models - Build Model Inference APIs, Job queues, LLM apps, Multi-model pipelines, and more!
https://bentoml.com
Apache License 2.0
7.17k stars 792 forks source link

bug: `bento containerize --opt platform=linux/amd64` fails #3440

Closed phitoduck closed 1 year ago

phitoduck commented 1 year ago

Describe the bug

My team uses Bitbucket Pipelines for CI/CD. Bitbucket Pipelines does not support Docker buildkit.

We were happy when the recent improvements to bentoml containerize ... came out in bentoml==1.0.11. We are no longer facing issues with buildkit, but now see this error instead.

First, trying to run

DOCKER_BUILDKIT=0 \
    BENTOML_DEBUG=TRUE \
    bentoml containerize {{BENTO_SERVICE_NAME}}:{{BENTO_SERVICE_TAG}} \
        --docker-image-tag {{BENTO_SERVICE_NAME}}:{{BENTO_SERVICE_TAG}} \
        --platform linux/amd64

I get a warning and a traceback:

'--platform=linux/amd64' is now obsolete, use the equivalent '--opt platform=linux/amd64' instead.
Building OCI-compliant image for keras-ctr-service:pkgj3oewworpdlg6 with docker

Traceback (most recent call last):
  File "/Users/eric/repos/ben/ctr-thumbnail-heatmap-api/venv/bin/bentoml", line 8, in <module>
    sys.exit(cli())
  File "/Users/eric/repos/ben/ctr-thumbnail-heatmap-api/venv/lib/python3.8/site-packages/click/core.py", line 1130, in __call__
    return self.main(*args, **kwargs)
  File "/Users/eric/repos/ben/ctr-thumbnail-heatmap-api/venv/lib/python3.8/site-packages/click/core.py", line 1055, in main
    rv = self.invoke(ctx)
  File "/Users/eric/repos/ben/ctr-thumbnail-heatmap-api/venv/lib/python3.8/site-packages/click/core.py", line 1657, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
  File "/Users/eric/repos/ben/ctr-thumbnail-heatmap-api/venv/lib/python3.8/site-packages/click/core.py", line 1404, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "/Users/eric/repos/ben/ctr-thumbnail-heatmap-api/venv/lib/python3.8/site-packages/click/core.py", line 760, in invoke
    return __callback(*args, **kwargs)
  File "/Users/eric/repos/ben/ctr-thumbnail-heatmap-api/venv/lib/python3.8/site-packages/bentoml_cli/utils.py", line 268, in wrapper
    return func(*args, **kwargs)
  File "/Users/eric/repos/ben/ctr-thumbnail-heatmap-api/venv/lib/python3.8/site-packages/bentoml_cli/utils.py", line 239, in wrapper
    return_value = func(*args, **kwargs)
  File "/Users/eric/repos/ben/ctr-thumbnail-heatmap-api/venv/lib/python3.8/site-packages/bentoml_cli/utils.py", line 196, in wrapper
    return func(*args, **kwargs)
  File "/Users/eric/repos/ben/ctr-thumbnail-heatmap-api/venv/lib/python3.8/site-packages/bentoml_cli/utils.py", line 55, in wrapper
    return _f(*args, **transformed)
  File "/Users/eric/repos/ben/ctr-thumbnail-heatmap-api/venv/lib/python3.8/site-packages/bentoml_cli/containerize.py", line 561, in containerize
    result = container.build(
  File "/Users/eric/repos/ben/ctr-thumbnail-heatmap-api/venv/lib/python3.8/site-packages/simple_di/__init__.py", line 139, in _
    return func(*_inject_args(bind.args), **_inject_kwargs(bind.kwargs))
  File "/Users/eric/repos/ben/ctr-thumbnail-heatmap-api/venv/lib/python3.8/site-packages/bentoml/container.py", line 404, in build
    passed_platform = ",".join(map(lambda v: v.split("=")[1], platform))
  File "/Users/eric/repos/ben/ctr-thumbnail-heatmap-api/venv/lib/python3.8/site-packages/bentoml/container.py", line 404, in <lambda>
    passed_platform = ",".join(map(lambda v: v.split("=")[1], platform))
IndexError: list index out of range

After making the suggested change, the warning goes away, but the traceback is identical:

DOCKER_BUILDKIT=0 \
    BENTOML_DEBUG=TRUE \
    bentoml containerize {{BENTO_SERVICE_NAME}}:{{BENTO_SERVICE_TAG}} \
        --docker-image-tag {{BENTO_SERVICE_NAME}}:{{BENTO_SERVICE_TAG}} \
        --opt platform=linux/amd64

Note: if I remove --opt platform=linux/amd64 and --platform linux/amd64 altogether, the build is successful. This workaround works, but is problematic for my team since many developers have M1 macs, so by default the built images are not compatible with CPUs in AWS and therefore, locally built images are impossible to test in the cloud.

To reproduce

My folder setup looks like this:

api/
├── bentofile.yaml
├── locustfile.py
├── pyproject.toml
├── setup.cfg
├── setup.py
├── src
│   ├── api
│   │   ├── __init__.py
│   │   ├── other modules ...
│   │   └── service.py
└── template.Dockerfile
# bentofile.yaml

service: api.service:svc

exclude:
  - ".gitignore"
  - "*.egg-info"
  - "*cache*"

python:
  # other packages are listed in setup.cfg, installed using the rendered template.Dockerfile (below)
  packages:
    - requests

docker:
  dockerfile_template: template.Dockerfile
  base_image: 763104351884.dkr.ecr.us-east-1.amazonaws.com/tensorflow-training:2.10.0-cpu-py39-ubuntu20.04-ec2

Expected behavior

The image should build successfully.

Environment

Environment variable

BENTOML_DEBUG=''
BENTOML_QUIET=''
BENTOML_BUNDLE_LOCAL_BUILD=''
BENTOML_DO_NOT_TRACK=''
BENTOML_CONFIG=''
BENTOML_CONFIG_OPTIONS=''
BENTOML_PORT=''
BENTOML_HOST=''
BENTOML_API_WORKERS=''

System information

bentoml: 1.0.12 python: 3.8.4 platform: macOS-10.16-x86_64-i386-64bit uid_gid: 502:20

pip_packages
``` absl-py==1.4.0 aiohttp==3.8.3 aiosignal==1.3.1 altair==4.2.0 anyio==3.6.2 -e git+ssh://git@bitbucket.org/bengrouptech/ctr-thumbnail-heatmap-api.git@154f557bef8c977a91e875df401f189859c22d4a#egg=api&subdirectory=api appdirs==1.4.4 appnope==0.1.3 argon2-cffi==21.3.0 argon2-cffi-bindings==21.2.0 arrow==1.2.3 asgiref==3.6.0 asttokens==2.2.1 astunparse==1.6.3 async-timeout==4.0.2 attrs==22.2.0 aws-cdk-lib==2.60.0 aws-cdk.asset-awscli-v1==2.2.49 aws-cdk.asset-kubectl-v20==2.1.1 aws-cdk.asset-node-proxy-agent-v5==2.0.38 aws-cdk.aws-apigatewayv2-alpha==2.60.0a0 aws-cdk.aws-apigatewayv2-authorizers-alpha==2.60.0a0 aws-cdk.aws-apigatewayv2-integrations-alpha==2.60.0a0 aws-cdk.aws-lambda-python-alpha==2.60.0a0 Babel==2.11.0 backcall==0.2.0 backoff==2.2.1 backports.zoneinfo==0.2.1 beautifulsoup4==4.11.1 bentoml==1.0.12 bleach==5.0.1 blinker==1.5 boto3==1.26.48 botocore==1.29.48 Brotli==1.0.9 build==0.10.0 cachetools==5.2.1 cattrs==22.2.0 certifi==2022.12.7 cffi==1.15.1 charset-normalizer==3.0.1 circus==0.18.0 click==8.1.3 click-option-group==0.5.5 cloudpickle==2.2.0 comm==0.1.2 commonmark==0.9.1 ConfigArgParse==1.5.3 constructs==10.1.217 contextlib2==21.6.0 contourpy==1.0.6 cycler==0.11.0 debugpy==1.6.5 decorator==5.1.1 deepmerge==1.1.0 defusedxml==0.7.1 Deprecated==1.2.13 entrypoints==0.4 exceptiongroup==1.1.0 executing==1.2.0 fastjsonschema==2.16.2 Flask==2.2.2 Flask-BasicAuth==0.2.0 Flask-Cors==3.0.10 flatbuffers==23.1.4 fonttools==4.38.0 fqdn==1.5.1 frozenlist==1.3.3 fs==2.4.16 gast==0.4.0 gevent==22.10.2 geventhttpclient==2.0.8 gitdb==4.0.10 GitPython==3.1.30 google-auth==2.16.0 google-auth-oauthlib==0.4.6 google-pasta==0.2.0 googleapis-common-protos==1.58.0 greenlet==2.0.1 grpcio==1.51.1 h11==0.14.0 h5py==3.7.0 -e git+ssh://git@bitbucket.org/bengrouptech/ctr-thumbnail-heatmap-api.git@154f557bef8c977a91e875df401f189859c22d4a#egg=iac&subdirectory=iac idna==3.4 imageio==2.24.0 importlib-metadata==6.0.0 importlib-resources==5.10.2 ipykernel==6.20.1 ipython==8.8.0 ipython-genutils==0.2.0 isoduration==20.11.0 itsdangerous==2.1.2 jedi==0.18.2 Jinja2==3.1.2 jmespath==1.0.1 joblib==1.2.0 jsii==1.73.0 json5==0.9.11 jsonpointer==2.3 jsonschema==4.17.3 jupyter-events==0.6.3 jupyter_client==7.4.9 jupyter_core==5.1.3 jupyter_server==2.0.6 jupyter_server_terminals==0.4.4 jupyterlab==3.5.2 jupyterlab-pygments==0.2.2 jupyterlab_server==2.18.0 keras==2.10.0 Keras-Preprocessing==1.1.2 kiwisolver==1.4.4 libclang==15.0.6.1 locust==2.14.2 Markdown==3.4.1 MarkupSafe==2.1.1 matplotlib==3.3.1 matplotlib-inline==0.1.6 mistune==2.0.4 msgpack==1.0.4 multidict==6.0.4 mypy==0.782 mypy-extensions==0.4.3 nbclassic==0.4.8 nbclient==0.7.2 nbconvert==7.2.7 nbformat==5.7.3 nest-asyncio==1.5.6 networkx==3.0 notebook==6.5.2 notebook_shim==0.2.2 numpy==1.24.1 oauthlib==3.2.2 opencv-python==4.6.0.66 opencv-python-headless==4.7.0.68 opentelemetry-api==1.14.0 opentelemetry-exporter-otlp-proto-http==1.14.0 opentelemetry-instrumentation==0.35b0 opentelemetry-instrumentation-aiohttp-client==0.35b0 opentelemetry-instrumentation-asgi==0.35b0 opentelemetry-proto==1.14.0 opentelemetry-sdk==1.14.0 opentelemetry-semantic-conventions==0.35b0 opentelemetry-util-http==0.35b0 opt-einsum==3.3.0 packaging==23.0 pandas==1.5.2 pandocfilters==1.5.0 parso==0.8.3 pathspec==0.10.3 pexpect==4.8.0 phitoduck-projen==0.1.1 pickleshare==0.7.5 Pillow==9.4.0 pip-requirements-parser==32.0.1 pip-tools==6.12.1 pkgutil_resolve_name==1.3.10 platformdirs==2.6.2 projen==0.66.12 prometheus-client==0.15.0 prompt-toolkit==3.0.36 protobuf==3.19.6 psutil==5.9.4 ptyprocess==0.7.0 publication==0.0.3 pure-eval==0.2.2 pyarrow==10.0.1 pyasn1==0.4.8 pyasn1-modules==0.2.8 pycparser==2.21 pydantic==1.10.4 pydeck==0.8.0 Pygments==2.14.0 Pympler==1.0.1 pynvml==11.4.1 pyparsing==3.0.9 pyproject_hooks==1.0.0 pyrsistent==0.19.3 python-dateutil==2.8.2 python-dotenv==0.21.0 python-json-logger==2.0.4 python-multipart==0.0.5 pytz==2022.7 pytz-deprecation-shim==0.1.0.post0 PyWavelets==1.4.1 PyYAML==6.0 pyzmq==25.0.0 requests==2.28.2 requests-oauthlib==1.3.1 rfc3339-validator==0.1.4 rfc3986-validator==0.1.1 rich==13.0.1 roundrobin==0.0.4 rsa==4.9 s3transfer==0.6.0 schema==0.7.5 scikit-image==0.19.3 scikit-learn==1.2.0 scipy==1.10.0 seaborn==0.12.2 semver==2.13.0 Send2Trash==1.8.0 simple-di==0.1.5 six==1.16.0 smmap==5.0.0 sniffio==1.3.0 soupsieve==2.3.2.post1 stack-data==0.6.2 starlette==0.23.1 streamlit==1.16.0 tensorboard==2.10.1 tensorboard-data-server==0.6.1 tensorboard-plugin-wit==1.8.1 tensorflow==2.10.0 tensorflow-addons==0.19.0 tensorflow-estimator==2.10.0 tensorflow-io-gcs-filesystem==0.29.0 termcolor==2.2.0 terminado==0.17.1 tf-keras-vis==0.8.2 threadpoolctl==3.1.0 tifffile==2022.10.10 tinycss2==1.2.1 toml==0.10.2 tomli==2.0.1 toolz==0.12.0 tornado==6.2 traitlets==5.8.1 typed-ast==1.4.3 typeguard==2.13.3 typing==3.7.4.3 typing_extensions==4.4.0 tzdata==2022.7 tzlocal==4.2 uri-template==1.2.0 urllib3==1.26.14 uvicorn==0.20.0 validators==0.20.0 watchfiles==0.18.1 wcwidth==0.2.5 webcolors==1.12 webencodings==0.5.1 websocket-client==1.4.2 Werkzeug==2.2.2 wrapt==1.14.1 yarl==1.8.2 zipp==3.11.0 zope.event==4.6 zope.interface==5.5.2 ```
phitoduck commented 1 year ago

This is possibly a duplicate of #3365. Although, #3365 says that the PR (#3366) would fix the issue. It's possible the fix isn't in 1.0.12 yet and that's the reason I'm still experiencing this error.

sauyon commented 1 year ago

I believe that's correct, we haven't done a release since that PR was merged.

sauyon commented 1 year ago

Going to close this as a duplicate of #3365 for now; I'll try to ping you again when we do a release.