squash the docker images before pushing them manually to docker.io and quay.io

radanalyticsio / openshift-spark

72 stars 83 forks source link

squash the docker images before pushing them manually to docker.io and quay.io #79

Closed jkremser closed 5 years ago

jkremser commented 5 years ago

This PR adds a script that for each image that we publish does docker-squash for the last 22 layers (23rd is the Centos base image layer) and pushes the image. For each commit to master it runs the same logic to create :latest images, for tags that matches the "regexp" x.y.z-n, it creates those image tags and also generates the x.y.z-latest so for instance if the git tag is 2.4.0-1, it will create and push images for :latest, 2.4.0-1 and also 2.4-latest.

profit if this is merged: images will be 300 megs smaller

todo:

[ ] set the login and password env variables either by travis secrets or directly in the travis config (I don't have rights for this)
[ ] turn-off the automatic dockerbuilds on docker.io and quay.io (this should be done after this is merged)
[ ] verify that it actually works as expected (requires the two above to be done)

/cc @elmiko

elmiko commented 5 years ago

i'm curious about

* turn-off the automatic dockerbuilds on docker.io and quay.io

this is because the new approach uses travis for doing the image builds?

jkremser commented 5 years ago

this is because the new approach uses travis for doing the image builds?

yes, we can't have both at the same time, it would override the images in the repository. Automated dockerbuilds are easy and quick to setup, but they don't provide the same amount of flexibility as travis+bash combo. It's not possible to run docker-squash for instance, or doing any tag magic.

tmckayus commented 5 years ago

This looks good to me, but I'm wondering about merge order. Should we do the spark 2.4 change first and make sure everything works before adding this in? And maybe docker and quay builds should be shut off before we merge this -- will the merge cause the travis build to fire?

elmiko commented 5 years ago

i think the main timing piece we need to worry about is the image autobuilds that are currently happening. it looked like the 2.4 stuff could happen on either side of this pr.

jkremser commented 5 years ago

I am ok w/ waiting for the other PR, however it could be a good oportunity to test the logic that's based on the tags in git.

will the merge cause the travis build to fire

once it's in master, it should trigger the build for the :latest images (this "if-branch")

jkremser commented 5 years ago

@elmiko I fixed that typo you've caught, however the PR still says "elmiko requested changes".

elmiko commented 5 years ago

@elmiko I fixed that typo you've caught, however the PR still says "elmiko requested changes".

sorry @Jiri-Kremser , i missed the email when the last change got pushed.

jkremser commented 5 years ago

:shipit: