Open Divyesh3011 opened 1 year ago
kcq
commented
1 year ago @Divyesh3011 what version of slim are you using? Also wonder why you disabled http probing: docker-slim build --target soc2-test-11 --tag soc2-test-slim --http-probe=false
Can you also describe what your app does and how the missing libraries are used in your app if you use them directly (e.g., pydantic)?
Divyesh3011
commented
1 year ago slim version linux|Transformer|1.40.2|762eea87bafc4b04fe6b0642a62534a148acdaca|2023-05-20_06:09:23AM there is no specific reason behind disabling http probing i just found one of the eg similar to this so i used this cmd should i modify the cmd ? the app is part of multiple microservices which are use to scan the image for any vulnerablity this is my requirement file i have multiple microservices most of them use the python:3.8-slim image as base & i am getting similar module/packages error can you help me with a standard docker-file for python svc that would support docker-slim requirement.txt
kcq
commented
1 year ago There's a couple of fastapi examples that might be useful for you: https://github.com/slimtoolkit/examples/tree/master/3rdparty/python_fastapi_official (and there's another one in the 3rdparty director, it has fastapi in its name too)
I thought you disabled http probing because you didn't have any EXPOSE instructions in your Dockerfile. You should have it, but if you don't you should use the --expose build command flag. Either of those ways tells slim the port to probe.
Divyesh3011
commented
1 year ago i tried updating my dockerfile according to your suggestion but getting this error i have added the slim-report file too cmd : docker-slim build --target azure-soc2 --tag azure-soc2:slim i have tried with this cmd too but getting the same error docker-slim build --target azure-soc2 --tag azure-soc2:slim --expose=8000
dockerfile FROM python:3.10-slim-buster WORKDIR /app COPY requirements.txt requirements.txt RUN apt-get -y update RUN pip3 install -r requirements.txt COPY . . EXPOSE 8000 CMD ["bash", "azure-soc2.sh"]
cmd=build info=container message='obtained IP address' ip='172.17.0.2' cmd=build info=cmd.startmonitor status='sent' cmd=build info=event.startmonitor.done status='received' cmd=build info=container name='slimk_27768_20230718143106' id='022295ba021c460c886551fbc3b3861caaee9f24cb9bc61b77b14f628423a5f1' target.port.list='49191' target.port.info='8000/tcp => 0.0.0.0:49191' message='YOU CAN USE THESE PORTS TO INTERACT WITH THE CONTAINER' cmd=build state=http.probe.starting message="WAIT FOR HTTP PROBE TO FINISH" cmd=build info=continue.after mode='probe' message='no input required, execution will resume when HTTP probing is completed'
cmd=build prompt='waiting for the HTTP probe to finish' cmd=build state=http.probe.running cmd=build info=http.probe.ports count='1' targets='49191' cmd=build info=http.probe.commands count='1' commands='GET /' cmd=build info=http.probe.call method='GET' target='http://127.0.0.1:49191/' attempt='1' error='Get "http://127.0.0.1:49191/": EOF' time='2023-07-18T14:31:20Z' status='error' cmd=build info=http.probe.call error='Get "http://127.0.0.1:49191/": EOF' time='2023-07-18T14:31:36Z' status='error' method='GET' target='http://127.0.0.1:49191/' attempt='2' cmd=build info=http.probe.call status='error' method='GET' target='http://127.0.0.1:49191/' attempt='3' error='Get "http://127.0.0.1:49191/": EOF' time='2023-07-18T14:31:52Z' cmd=build info=http.probe.call attempt='4' error='Get "http://127.0.0.1:49191/": EOF' time='2023-07-18T14:32:08Z' status='error' method='GET' target='http://127.0.0.1:49191/' cmd=build info=http.probe.call status='error' method='GET' target='http://127.0.0.1:49191/' attempt='5' error='Get "http://127.0.0.1:49191/": EOF' time='2023-07-18T14:32:24Z' cmd=build info=http.probe.summary successful='0' total='5' failures='5' cmd=build state=http.probe.done warning=no.successful.calls cmd=build info=event message='HTTP probe is done' cmd=build error=probe.error message='no.successful.calls' slim: container stdout:
kcq
commented
1 year ago Can you describe what your app does?
Divyesh3011
commented
1 year ago the app is part of multiple microservices which are use to scan the image for any vulnerablity i is there are way where i can specify the slim not to delete certain package like having a preserve.txt file where we keep all the required package & dependecy & tell slim to make sure all the package specified in the preserve.txt file are present in final Image i have multiple microservice & for different svc i get different/package depedency error so having something like preserve.txt file will help to solve the errors
Docker file FROM python:3.10-slim-buster RUN pip3 install anyio==3.5.0 WORKDIR /app COPY requirements.txt requirements.txt RUN pip3 install --no-cache-dir -r requirements.txt COPY . . EXPOSE 8000 CMD ["python", "start_server.py", "--reload","--port=8000"]
one of the depedecy-error i am facing (this dependecy is mentioned in requirement.txt file) with normal image it work but with slim image i get errors
return _bootstrap._gcd_import(name[level:], package, level)
File "
Divyesh3011
commented
1 year ago @kcq
Traceback (most recent call last):
File "/app/start_server.py", line 1, in
It shows no app.models.discovery but i have checked the path it contains the file /app └── app ├── init.py ├── main.py └── models ├── init.py └── discovery.py
docker-slim build --target gcr.io/***/aws-discovery-v2-dump:no-slim --tag gcr.io/***/aws-discovery-v2-dump:slim1 --http-probe=true --continue-after=40 --include-shell=true --include-path /usr/local/lib/
Facing Modules / package error
2023-07-12 17:08:15 Traceback (most recent call last): 2023-07-12 17:08:15 File "/usr/local/lib/python3.8/decimal.py", line 3, in
2023-07-12 17:08:15 from _decimal import
2023-07-12 17:08:15 ModuleNotFoundError: No module named 'numbers'
2023-07-12 17:08:15
2023-07-12 17:08:15 During handling of the above exception, another exception occurred:
2023-07-12 17:08:15
2023-07-12 17:08:15 Traceback (most recent call last):
2023-07-12 17:08:15 File "main.py", line 2, in
2023-07-12 17:08:15 from fastapi import FastAPI
2023-07-12 17:08:15 File "/usr/local/lib/python3.8/site-packages/fastapi/init.py", line 7, in
2023-07-12 17:08:15 from .applications import FastAPI as FastAPI
2023-07-12 17:08:15 File "/usr/local/lib/python3.8/site-packages/fastapi/applications.py", line 3, in
2023-07-12 17:08:15 from fastapi import routing
2023-07-12 17:08:15 File "/usr/local/lib/python3.8/site-packages/fastapi/routing.py", line 20, in
2023-07-12 17:08:15 from fastapi import params
2023-07-12 17:08:15 File "/usr/local/lib/python3.8/site-packages/fastapi/params.py", line 4, in
2023-07-12 17:08:15 from pydantic.fields import FieldInfo, Undefined
2023-07-12 17:08:15 File "pydantic/init.py", line 2, in init pydantic.init
2023-07-12 17:08:15 File "pydantic/dataclasses.py", line 3, in init pydantic.dataclasses
2023-07-12 17:08:15 import copy
2023-07-12 17:08:15 File "pydantic/class_validators.py", line 8, in init pydantic.class_validators
2023-07-12 17:08:15 File "pydantic/errors.py", line 1, in init pydantic.errors
2023-07-12 17:08:15 File "/usr/local/lib/python3.8/decimal.py", line 8, in
2023-07-12 17:08:15 from _pydecimal import
2023-07-12 17:08:15 ModuleNotFoundError: No module named '_pydecimal'
Steps to Reproduce the Problem (DockerFile)
FROM python:3.8-slim-buster AS build WORKDIR /app COPY . . RUN apt-get update && apt-get install -y --no-install-recommends \ build-essential \ && rm -rf /var/lib/apt/lists/* RUN pip install --no-cache-dir -r requirements.txt
Stage 2: Final stage
FROM python:3.8-slim-buster WORKDIR /app COPY --from=build /usr/local/lib/python3.8/site-packages /usr/local/lib/python3.8/site-packages COPY . . RUN pip install --no-cache-dir -r requirements.txt fastapi==0.68.1 uvicorn==0.15.0 CMD ["bash", "azure-soc2-manager.sh"] EXPOSE 8000
Create preserved-paths.txt file in the final image
RUN echo "/usr/local/usr/local/lib/python3.8" > /preserved-paths.txt
docker images | grep soc2
soc2-test-1 slim cf086482e0b3 About a minute ago 33.9MB soc2-test-1 latest d3fdfab4467d 10 minutes ago 279MB