search

quarkusio / quarkus

Quarkus: Supersonic Subatomic Java.
https://quarkus.io
Apache License 2.0
13.74k stars 2.67k forks source link

Error starting native application container in distroless docker image #8974

Closed serrodcal closed 1 month ago

serrodcal commented 4 years ago

Describe the bug I have an application which I want to build in a distroless Docker image. I follow this README and the container doesn't start.

Expected behavior The container should start fine.

Actual behavior The container doesn't start getting ´standard_init_linux.go:211: exec user process caused "exec format error"´ error message.

To Reproduce Steps to reproduce the behavior:

  1. Go to project root folder
  2. Build it with ./mvnw clean package -Pnative -Dnative-image.docker-build=true as I can read in the README above.
  3. Build the Docker image with: docker build -f src/main/docker/Dockerfile.distroless -t employee-distroless:1.0.1 ..
  4. Finally, try to start the container with: docker run -i --rm -p 8080:8080 employee-distroless:1.0.1.

Environment (please complete the following information):

geoand commented 4 years ago

cc @cescoffier

serrodcal commented 4 years ago

I also tested with -Dquarkus.native.container-runtime=docker but It doesn't work either.

serrodcal commented 4 years ago

Hi, this is not a big deal but I'd like to know if you plan to fix this issue or not. It'd be interesting to be able to build distroless images to reduce the download time.

cescoffier commented 4 years ago

I didn't touch the distroless part for a long time. I will retry soon.

serrodcal commented 4 years ago

Yeah, I saw the repository as well. Don't worry, this just a kindly reminder to improve that part. Any help from my side just tell me, because I have "realistic" application to test distroless images.

Thanks!

serrodcal commented 4 years ago

Hi again, I was trying to figure out which dependency brakes the distroless option. Finally, I found 'quarkus-smallrye-opentracing' dependency results in ./application: error while loading shared libraries: libstdc++.so.6: cannot open shared object file: No such file or directory.

It looks like the system needs the 'libstdc++6' library (according this link -> https://askubuntu.com/questions/409821/libstdc-so-6-cannot-open-shared-object-file-no-such-file-or-directory)

If I find any new information regarding this, I will add in this issue.

Hope this help you Quarkus Team to build better framework :)

cescoffier commented 4 years ago

Definitely! Thanks!

angelfretz23 commented 4 years ago

@serrodcal I had the same issue. I was able to solve it by using -Dquarkus.native.container-build=true instead of -Dnative-image.docker-build=true during the packaging phase. Im not sure if it will solve the issue for you but worth a try

serrodcal commented 4 years ago

@serrodcal I had the same issue. I was able to solve it by using -Dquarkus.native.container-build=true instead of -Dnative-image.docker-build=true during the packaging phase. Im not sure if it will solve the issue for you but worth a try

I will try it, but I think the error is related to a library is missing... Thanks Angel.

angelfretz23 commented 4 years ago

@serrodcal Sorry, it was very late (2:20 am my time). I should have explained that when i used -Dquarkus.native.container-build=true, Docker downloaded a dependency that it didnt when I used -Dnative-image.docker-build=true so it might be the missing library =)

serrodcal commented 4 years ago

@serrodcal Sorry, it was very late (2:20 am my time). I should have explained that when i used -Dquarkus.native.container-build=true, Docker downloaded a dependency that it didnt when I used -Dnative-image.docker-build=true so it might be the missing library =)

Hi Angel, thanks again for your help.

The problem is still there, ./application: error while loading shared libraries: libstdc++.so.6: cannot open shared object file: No such file or directory. Just to clarify, I get that error in a native application within a distroless container. And, as far as I know, the library is not within the docker base image.

This is the project I'm trying to build in native+distroless, just if you want to try it. The maven command I used was ./mvnw clean package -Pnative -Dnative-image.docker-build=true -Dquarkus.native.container-runtime=docker -Dquarkus.native.container-build=true.

Also, I tried to build my own distroless base image using bazel but I couldn't. I put my error building that image in a separated issue. Let me know if you are interesting on it to look up the url.

maxday commented 3 years ago

Hello I have managed to build a distroless image :)

Here is the code :

FROM registry.access.redhat.com/ubi8/ubi-minimal as nativebuilder   
RUN mkdir -p /tmp/ssl \
&& cp /usr/lib64/libstdc++.so.6.0.25 /tmp/ssl/libstdc++.so.6 \
&& cp /usr/lib64/libgcc_s-8-20191121.so.1 /tmp/ssl/libgcc_s.so.1 \
&& cp /usr/lib64/libz.so.1 /tmp/ssl/libz.so.1

FROM gcr.io/distroless/base
COPY --from=nativebuilder /tmp/ssl/ /
ENV LD_LIBRARY_PATH /

COPY target/*-runner /application
EXPOSE 8080
CMD ["./application", "-Dquarkus.http.host=0.0.0.0"]

@cescoffier What do you think about this approach ? I could document it here if you want : https://github.com/quarkusio/quarkus-images/tree/master/distroless

Here is a working example : https://github.com/maxday/quarkus-native-distroless

cescoffier commented 3 years ago

Nice @maxday! I like the idea to use another container to extract the .so files.

serrodcal commented 3 years ago

Tested and working fine!

Thanks @maxday

geoand commented 3 years ago

I also think the builder approach is nice 👍🏼

maxday commented 3 years ago

Thanks ! I'm going to create a PR for this this afternoon

maxday commented 3 years ago

oh I see that @matthyx is doing exactly that here https://github.com/quarkusio/quarkus-images/pull/118

matthyx commented 3 years ago

Yeah, actually it's better to take out the libs from debian as it is what they use for the distroless images. I had to shoehorn a bit my ideas into cekit. Let's merge and enjoy!

kamilgregorczyk commented 3 years ago

I think the difference I that when you build ./gradlew build -Dquarkus.package.type=native on a Mac, it will compile it to Mac byte code, which then after being copied to docker is no longer compatible with linux. By doing ./gradlew build -Dquarkus.package.type=native -Dquarkus.native.container-build=true you are actually building it with docker for linux which makes the binary no longer runnable on Mac but it runs after copying to linux.

geoand commented 1 month ago

Is this still an issue?

cescoffier commented 1 month ago

I think the comments above show the solution. Not much we can do (Except maybe document, but I'm not sure we document distroless, as UBI-Micro is just a few more Mb without the burden).

geoand commented 1 month ago

Let's close in that case