Switch to distroless java11 base image.

anuraaga commented 5 years ago

Fixes #197 see the issue for the image sizes.

If we get a lot of boringssl-related issues, using this image could help since we can disable boringssl and still have HTTP/2. Though since it uses glibc instead of musl, maybe random native library issues would be less frequent anyways.

Moved the command into a script since I was having trouble getting the syntax to be accepted by busybox and it's arcane enough to have its own script I guess.

I only really tested with the defaults, not with any of the environment variable knobs set.

codefromthecrypt commented 5 years ago

thanks I will try to build it now!

codefromthecrypt commented 5 years ago

this works.. just pondering if we shouldn't make this a base image instead as it will help the total GB's pulled when people are doing things like docker-compose off a hotel or conference net

anuraaga commented 5 years ago

Migrated most of the images and came up with these findings

Cassandra and Elasticsearch 2 are very old and don't run in default configurations on Java 11. Some config file hacking of GC settings got them to run though
Still need to work more on the kafka image. While busybox includes runit, it's giving a weird error that has me stumped for now. What do you think about separating out the zookeeper image and connecting with docker-compose instead of including it in the kafka image itself? I think it's generally considered bad practice to start two servers in a single container
MySQL would be very difficult to run on distroless. Distroless is great for Java binaries since you usually just download and run them, but MySQL is installed through the distribution package manager - distroless has no distribution ;) The common way of adding to a distroless is to install into debian:stretch and copy into the distroless image, but this would be very tricky for MySQL which has lots of dependent packages. I might still be able to get it to work but wonder how important it is

codefromthecrypt commented 5 years ago

thanks for the update. only stronger opinion is on kafka+zk this is not a production image by the way.. none except zipkin are. people have so many problems with Kafka setup I really prefer to not split the image only to have a new possible problem ;)

anuraaga commented 5 years ago

Ok I think I'll play with the Kafka a bit more. Do you remember why we use runit and not just run the commands directly? Do Kafka and Zookeeper just randomly shutdown a lot and need to be restarted within the container? Ugh ;)

codefromthecrypt commented 5 years ago

there is a race condition on ZK startup. iirc Kafka gets really upset if ZK is down

this is the thing I don't want first time users to end up having to think too much about ;)

On Tue, Feb 19, 2019, 7:05 PM Anuraag Agrawal <notifications@github.com wrote:

Ok I think I'll play with the Kafka a bit more. Do you remember why we use runit and not just run the commands directly? Do Kafka and Zookeeper just randomly shutdown a lot and need to be restarted within the container? Ugh ;)

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/openzipkin/docker-zipkin/pull/199#issuecomment-465087482, or mute the thread https://github.com/notifications/unsubscribe-auth/AAD61xvvn8aNVjdnTawkOe296P4WSFjZks5vO9p4gaJpZM4a8_MB .

anuraaga commented 5 years ago

Ok think I got everything working

Just use normal commands to start up Zookeeper + Kafka with a wait in between. This seems to work better than runit since now I can Ctrl+C the container instead of having to docker ps docker kill
Don't download zookeeper into kafka image. Apparently kafka comes with zookeeper built-in. So some size savings for free IIUC
Use alpine as base image for mysql since mysql doesn't need a JRE

This means there are two base images involved now, one that includes JRE11 based on distroless for all the Java stuff and one that is alpine which is tiny so hopefully is not too much of a bummer.

codefromthecrypt commented 5 years ago

wow.. sounds like you are ready for me to test. is that right?

anuraaga commented 5 years ago

Yup think it should be ready to test

codefromthecrypt commented 5 years ago

I verified that the elasticsearch and zipkin images use the same base layers ( building with docker-compose). It is a shame that the JRE is 170MiB, but that's

○ → docker history docker-zipkin_storage
IMAGE               CREATED              CREATED BY                                      SIZE                COMMENT
d33934fd4fc5        About a minute ago   /bin/sh -c #(nop)  ENTRYPOINT ["/busybox/sh"…   0B                  
2fb3aa6c73ea        About a minute ago   /bin/sh -c #(nop)  ENV JAVA_OPTS=               0B                  
495a3be72d56        About a minute ago   /bin/sh -c #(nop)  EXPOSE 9200 9300             0B                  
ec52c15b8a87        About a minute ago   /bin/sh -c #(nop) COPY dir:f5faa329bc0b2389a…   84B                 
a8f0c979b1d5        About a minute ago   /bin/sh -c #(nop)  USER elasticsearch           0B                  
ff1e93efcda1        About a minute ago   /bin/sh -c #(nop) WORKDIR /elasticsearch        0B                  
c456a20f2c29        About a minute ago   /bin/sh -c #(nop) COPY --chown=elasticsearch…   168MB               
7baa3c7f7cce        About a minute ago   /busybox/sh -c adduser -g '' -D elasticsearch   163B                
6a60de9502c6        About a minute ago   /bin/sh -c #(nop)  ENV ES_TMPDIR=/tmp           0B                  
a0d4bcc63e07        About a minute ago   /bin/sh -c #(nop)  ENV ELASTICSEARCH_VERSION…   0B                  
ce5329e75a7d        2 weeks ago          /bin/sh -c #(nop)  MAINTAINER OpenZipkin "ht…   0B                  
d24b7bc16c49        49 years ago         bazel build ...                                 170MB               
<missing>           49 years ago         bazel build ...                                 1.93MB              
<missing>           49 years ago         bazel build ...                                 924kB               
<missing>           49 years ago         bazel build ...                                 15.1MB              
<missing>           49 years ago         bazel build ...                                 1.81MB              

 2019-03-03 07:22:09 ⌚ |ruby-2.6.0| MacBook-Pro-7 in ~
○ →  docker history docker-zipkin_zipkin 
IMAGE               CREATED             CREATED BY                                      SIZE                COMMENT
f1aeea6222bc        7 minutes ago       /bin/sh -c #(nop)  ENTRYPOINT ["/busybox/sh"…   0B                  
e26aa4e13647        7 minutes ago       /bin/sh -c #(nop)  EXPOSE 9410 9411             0B                  
dd3eca3e75c7        7 minutes ago       /bin/sh -c #(nop) WORKDIR /zipkin               0B                  
2718c93a45c0        7 minutes ago       /bin/sh -c #(nop) COPY dir:738823f7719cb3820…   56.2MB              
148b65a44806        2 weeks ago         /bin/sh -c #(nop)  ENV JAVA_OPTS=-Djava.secu…   0B                  
d27324f60de1        2 weeks ago         /bin/sh -c #(nop)  ENV ZIPKIN_VERSION=2.12.1    0B                  
ce5329e75a7d        2 weeks ago         /bin/sh -c #(nop)  MAINTAINER OpenZipkin "ht…   0B                  
d24b7bc16c49        49 years ago        bazel build ...                                 170MB               
<missing>           49 years ago        bazel build ...                                 1.93MB              
<missing>           49 years ago        bazel build ...                                 924kB               
<missing>           49 years ago        bazel build ...                                 15.1MB              
<missing>           49 years ago        bazel build ...                                 1.81MB

I also verified that the mysql image heft is in our realm to control :). It isn't optimized at all (in our Dockerfile and before this changed)

○ → docker history docker-zipkin_storage
IMAGE               CREATED             CREATED BY                                      SIZE                COMMENT
ad3142e896c0        10 minutes ago      /bin/sh -c #(nop)  EXPOSE 3306                  0B                  
73c5698585e0        10 minutes ago      /bin/sh -c #(nop)  CMD ["/bin/sh" "-c" "/mys…   0B                  
c46cd4472105        10 minutes ago      /bin/sh -c #(nop) ADD file:8c0d23bbfd11be3a7…   82B                 
6b5b19d7c83b        10 minutes ago      /bin/sh -c /mysql/configure                     127MB               
dd90c5d19052        10 minutes ago      /bin/sh -c #(nop) ADD file:8be946672fb026031…   1.14kB              
9ff6f8879c04        10 minutes ago      /bin/sh -c #(nop)  ENV ZIPKIN_VERSION=2.12.3    0B                  
2e355c70dc0e        10 minutes ago      /bin/sh -c /mysql/install                       264MB               
0cc82bc2f199        10 minutes ago      /bin/sh -c #(nop) ADD file:0418aa65ee2166469…   220B                
4e5f8e41ab4b        10 minutes ago      /bin/sh -c #(nop) WORKDIR /mysql                0B                  
8c800417b143        10 minutes ago      /bin/sh -c #(nop)  MAINTAINER OpenZipkin "ht…   0B                  
caf27325b298        4 weeks ago         /bin/sh -c #(nop)  CMD ["/bin/sh"]              0B                  
<missing>           4 weeks ago         /bin/sh -c #(nop) ADD file:2a1fc9351afe35698…   5.53MB

codefromthecrypt commented 5 years ago

still getting the following running with docker-compose. I'll try to fix it

zipkin                      | run.sh: source: line 5: ._profile: not found

codefromthecrypt commented 5 years ago

update: I've tested all images by building local except the elasticsearch ones.. doing that now

codefromthecrypt commented 5 years ago

ok tested all images. we're good. understood that the zipkin-dependencies image can't use distroless as it is pinned to JDK 1.8

codefromthecrypt commented 5 years ago

Thanks again @anuraaga you rock!

openzipkin-attic / docker-zipkin

Switch to distroless java11 base image. #199