orfeas-k
commented
9 months ago After all, starlette package was indeed causing the issue. I managed to pin starlette by adding it to the requirements/docker.txt file before running pip install with it. This resolved the issue.
Regarding
Somehow, I have a .rock file that is working as expected. I 'm not sure from where exactly this was built although it uses mlserver package 1.3.5.
it turns out that rockcraft clean && rockcraft pack didn't install the same dependencies each time. This can be verified by those two CI runs that are built from the same commit (one from PR and one from merging the PR to main) and they have different packages installed (starlette and fsspec). One image works (as expected) and the other one doesn't. Here's diff between their pip freeze output
- fsspec==2023.10.0
+ fsspec==2023.12.2
- starlette==0.22.0
+ starlette==0.34.0
(this issue doesn't apply to
mlserver-huggingface.) While updating themlserver-*ROCKs to MLServer version 1.3.5 (for Seldon 1.17.1), the ROCKs started failing with error'FastAPI' object has no attribute 'debug'. Note that FastApi package is pinned in MLServer images.Debugging
Googling this prompts me that this has to do with
starlettepackage installed https://github.com/tiangolo/fastapi/discussions/5977. However,starletteisn't pinned anywhere in the MLServer repo, considering also the things we install in our ROCK.Somehow, I have a
.rockfile that is working as expected. I 'm not sure from where exactly this was built although it uses mlserver package 1.3.5. Comparing the twopipfreezes, it looks like the "working" one uses starlette package version of0.22.0. However, when I tried to downgrade it inside the image's command modifying it to the below (essentially addingpip install --prefix opt/mlserver/.local starlette==0.22.0)the image returns the error
On top of that, I tried to remove the pip install requirements/docker.txt line from the rockcraft.yaml file, which ended up with
starlette==0.22.0but the produced image still returned'FastAPI' object has no attribute 'debug'.Here are the
pip freezeresults from the aforementioned images (frommlserver-sklearn) pip-freeze-1.3.5.txt pip-freeze-1.3.5-no-pip-install-requirements-line.txt pip-freeze-1.3.5-working.txtComparing also the
.tarfiles using google-container-diff from the ROCK that fails and the one that returns expected results, the only remove/add difference I see is about the starlette package.The rest of the output mentions that all the other files that are different between the two images.
Full error logs
Full error logs
```bash ╰─$ docker run mlserver-sklearn:1.3.5 130 ↵ 2024-01-09T13:58:01.674Z [pebble] Started daemon. 2024-01-09T13:58:01.679Z [pebble] POST /v1/services 3.752579ms 202 2024-01-09T13:58:01.679Z [pebble] Started default services with change 1. 2024-01-09T13:58:01.682Z [pebble] Service "mlserver-sklearn" starting: bash -c 'cd /opt/mlserver && export PATH=/opt/conda/bin/:/opt/mlserver/.local/bin:${PATH}:/usr/bin && export PYTHONPATH=/opt/mlserver/.local/lib/python3.8/site-packages/:${PYTHONPATH} && eval $(/opt/conda/bin/conda shell.bash hook 2> /dev/null) && mlserver start ${MLSERVER_MODELS_DIR}' 2024-01-09T13:58:02.637Z [mlserver-sklearn] Traceback (most recent call last): 2024-01-09T13:58:02.637Z [mlserver-sklearn] File "/opt/mlserver/.local/bin/mlserver", line 8, in