Open jannichorst opened 4 months ago
@jannichorst thanks for the extra context and for the snippets! trying to repro it
@jannichorst What's needed is a custom http probe in the slim command. I will share what it should look like with my repro.
@kcq Thanks Kyle! Looking forward to the example. It really has been bugging me not getting it done myself...
Just out of curiosity, are you guys open/looking for contributions of examples? Asking because if I get it done I wouldn't mind giving back by providing another example... Let me know if that is something that would be of value to the project.
@jannichorst yes, definitely will be valuable! Any contribution will be great
Here's a slightly hacky way to make it work (it's including the pillow/PIL package directory explicitly):
building the image separately just to keep it simple:
docker build -t repro-python-fastapi-pillow .
minify the target image:
mint slim --http-probe-cmd /docs --http-probe-apispec /openapi.json --include-path /usr/local/lib/python3.9/site-packages/PIL --include-exe tesseract repro-python-fastapi-pillow
Make sure to use the latest release
The next upcoming release will not require you to have --include-path /usr/local/lib/python3.9/site-packages/PIL
. The http probes will handle multipart form uploads properly.
@kcq thanks for the example! I ran the commands and get the following output:
slim version
cmd=version info=app version='darwin/arm64|Transformer|1.40.11|latest|latest' container='false' dsimage='false' location='/opt/homebrew/bin' current='unknown' verdict='version status information is not available' cmd='version'
cmd=version info=host release='23.3.0' sysname='darwin' cmd='version' osname='Sonoma (14.3)' osbuild='23D56' version=' Wed Dec 20 21:30:44 PST 2023; root:xnu-10002.81.5~7/RELEASE_ARM64_T6000'
cmd=version info=docker ostype='linux' server.version='24.0.7' architecture='aarch64' cmd='version' name='docker-desktop' kernel.version='6.5.11-linuxkit' operating.system='Docker Desktop'
cmd=version info=dclient build.time='2023-10-26T09:08:15.000000000+00:00' git.commit='311b9ff' cmd='version' api.version='1.43' min.api.version='1.12'
mint version
cmd=version info=app dsimage='false' location='/usr/local/bin' current='1.41.2' verdict='you have the latest version' cmd='version' version='darwin/amd64|Aurora|1.41.2|d6f9778dcd40dfd2108737070b85d929c9806de6|2024-05-30_03:14:56PM' container='false'
cmd=version info=host osname='Sonoma (14.3)' osbuild='23D56' version=' Wed Dec 20 21:30:44 PST 2023; root:xnu-10002.81.5~7/RELEASE_ARM64_T6000' release='23.3.0' sysname='darwin' cmd='version'
cmd=version info=docker server.version='24.0.7' architecture='aarch64' cmd='version' name='docker-desktop' kernel.version='6.5.11-linuxkit' operating.system='Docker Desktop' ostype='linux'
cmd=version info=dclient api.version='1.43' min.api.version='1.12' build.time='2023-10-26T09:08:15.000000000+00:00' git.commit='311b9ff' cmd='version'
sudo mint slim --http-probe-cmd /docs --http-probe-apispec /openapi.json --include-path /usr/local/lib/python3.9/site-packages/PIL --include-exe tesseract repro-python-fastapi-pillow
cmd=slim state=started
cmd=slim info=cmd.input.params target.type='image' target.image='repro-python-fastapi-pillow' continue.mode='probe' rt.as.user='true' keep.perms='true' tags='' image-build-engine='internal'
cmd=slim state=image.inspection.start
cmd=slim info=image id='sha256:de16ad88404836cfbf51873d8144723dc325270c3f4a4f2a25db7ffe6150bebd' size.bytes='287296525' size.human='287 MB'
cmd=slim info=image.stack name='repro-python-fastapi-pillow:latest' id='sha256:de16ad88404836cfbf51873d8144723dc325270c3f4a4f2a25db7ffe6150bebd' index='0'
cmd=slim info=image.exposed_ports list='8000/tcp'
cmd=slim state=image.inspection.done
cmd=slim state=container.inspection.start
cmd=slim info=container status='created' name='mintk_3993_20240607063300' id='1fa920da9f1076093a76bb44fe6e86f19ee7bde06e6a268cda558f76dc694297'
cmd=slim info=container status='running' name='mintk_3993_20240607063300' id='1fa920da9f1076093a76bb44fe6e86f19ee7bde06e6a268cda558f76dc694297'
cmd=slim info=container message='obtained IP address' ip='172.17.0.2'
cmd=slim info=cmd.startmonitor status='sent'
cmd=slim info=event.startmonitor.done status='received'
cmd=slim info=container name='mintk_3993_20240607063300' id='1fa920da9f1076093a76bb44fe6e86f19ee7bde06e6a268cda558f76dc694297' target.port.list='55211' target.port.info='8000/tcp => 0.0.0.0:55211' message='YOU CAN USE THESE PORTS TO INTERACT WITH THE CONTAINER'
cmd=slim state=http.probe.starting message="WAIT FOR HTTP PROBE TO FINISH"
cmd=slim info=continue.after mode='probe' message='no input required, execution will resume when HTTP probing is completed'
cmd=slim prompt='waiting for the HTTP probe to finish'
cmd=slim state=http.probe.running
cmd=slim info=http.probe.ports count='1' targets='55211'
cmd=slim info=http.probe.commands count='1' commands='GET /docs'
cmd=slim info=http.probe.call status='error' method='GET' target='http://127.0.0.1:55211/docs' attempt='1' error='Get "http://127.0.0.1:55211/docs": read tcp 127.0.0.1:64409->127.0.0.1:55211: read: connection reset by peer' time='2024-06-07T06:33:15Z'
cmd=slim info=http.probe.call status='error' method='GET' target='http://127.0.0.1:55211/docs' attempt='2' error='Get "http://127.0.0.1:55211/docs": read tcp 127.0.0.1:64410->127.0.0.1:55211: read: connection reset by peer' time='2024-06-07T06:33:23Z'
cmd=slim info=http.probe.call target='http://127.0.0.1:55211/docs' attempt='3' error='Get "http://127.0.0.1:55211/docs": EOF' time='2024-06-07T06:33:31Z' status='error' method='GET'
cmd=slim info=http.probe.call error='Get "http://127.0.0.1:55211/docs": read tcp 127.0.0.1:64412->127.0.0.1:55211: read: connection reset by peer' time='2024-06-07T06:33:47Z' status='error' method='GET' target='http://127.0.0.1:55211/docs' attempt='4'
cmd=slim info=http.probe.call method='GET' target='http://127.0.0.1:55211/docs' attempt='5' error='Get "http://127.0.0.1:55211/docs": read tcp 127.0.0.1:64413->127.0.0.1:55211: read: connection reset by peer' time='2024-06-07T06:33:55Z' status='error'
cmd=slim info=http.probe.summary total='5' failures='5' successful='0'
cmd=slim state=http.probe.done warning=no.successful.calls
cmd=slim info=event message='HTTP probe is done'
cmd=slim error=probe.error message='no.successful.calls'
mint: container stdout:
mint: container stderr:
mint: end of container logs =============
cmd=slim state=exited code=-1
cmd=slim info=container.inspector.cleanup name='mintk_3993_20240607063300' id='1fa920da9f1076093a76bb44fe6e86f19ee7bde06e6a268cda558f76dc694297'
cmd=slim state=container.target.shutdown.start
cmd=slim state=container.target.shutdown.done
cmd=slim info=report file='slim.report.json'
cmd=slim info=exit code='-1' version='darwin/amd64|Aurora|1.41.2|d6f9778dcd40dfd2108737070b85d929c9806de6|2024-05-30_03:14:56PM' location='/usr/local/bin'
app='mint' message='GitHub Discussions' info='https://github.com/mintoolkit/mint/discussions'
app='mint' message='Join the CNCF Slack channel to ask questions or to share your feedback' info='https://cloud-native.slack.com/archives/C059QP1RH1S'
app='mint' message='Join the Discord server to ask questions or to share your feedback' info='https://discord.gg/fAvq4ruKsG'
Using the same files as before... Am I missing something? Or is there anything else I have to check for updates?
The new release (1.41.3)
now shouldn't need the --include-path
flag for PIL
.
This command should now produced a working image: mint slim --http-probe-full --http-probe-cmd /docs --http-probe-apispec /openapi.json YOUR_TARGET_IMAGE
Alternatively this should produce the same results: mint slim --http-probe-full --http-probe-cmd /docs --http-probe-cmd-upload generate.image:/upload YOUR_TARGET_IMAGE
The output you shared looks a bit odd because it shows that all http probe commands failed. Just in case, make sure to delete all sensor volumes. Do docker volume ls
and then do docker volume rm AAA
where AAA
is mint-sensor.1.41.2
or something similar.
It'll also help to look at the logs for the temporary container mint
creates (when you do docker ps
that's the containers with themintk_
names, so you can do docker logs mintk_AAA
). Adding the --debug
flag should provide extra info too (e.g., mint --debug slim --http-probe-full --http-probe-cmd /docs --http-probe-apispec /openapi.json YOUR_TARGET_IMAGE
).
Expected Behavior
After slimming a container, Pillow throws a "PIL.UnidentifiedImageError" when reading an image. The code below is for an endpoint, but the behaviour is the same when reading from the file system. The code by itself is fine, as the not-slimmed container does work just fine. Similarly, reading an image from the file system using pillow that is not part of an FastAPI application works too in a slimmed container. Something about the combination slimmed container + FastAPI seems to break pillow.
Am I missing any shared libraries I have to specify manually or what could be the issue? Thanks already for the support!
Actual Behavior
Steps to Reproduce the Problem
1. paste code above in main.py 2. execute: `slim --crt-api-version=1.25 build --dockerfile Dockerfile --include-exe tesseract --include-path /app --tag-fat test-container .` (don't forget to delete the ".txt" from the attached Dockerfile...) 3. start container with `docker run -it -p 8000:8000 test-container.slim` 4. open [http://0.0.0.0:8000/docs](http://0.0.0.0:8000/docs) in the browser 5. upload an image (jpeg/png) in the `/upload` section of the swagger documentation --- Specifications ================= - Version: 1.40.11 - Platform: MacOS ARM [Dockerfile.txt](https://github.com/user-attachments/files/15518966/Dockerfile.txt) [requirements.txt](https://github.com/user-attachments/files/15518967/requirements.txt)