DockerPlugin should pull for newest base image by default

wsargent commented 6 years ago

Expected behaviour

sbt docker:publishLocal gets the correct base image from remote if a newer one is available

Actual behaviour

sbt docker:publishLocal will use the local image if it is cached, will not pull the base image remotely even if it's newer

see https://www.databasesandlife.com/docker-build-pull-option/ for more details

Information

Using sbt-native-packager 1.3.2 on MacOS.

This is something that can be easily fixed by adding the --pull option to always check if there is a newer base image available:

dockerBuildOptions ++= Seq("--pull")

So maybe we can make that the default.

muuki88 commented 6 years ago

Thanks for your detailed issue along with a possible solution :smiley:

My day-to-day docker experience is limited, so I have a few questions :wink:

Can you still build your docker image if you are offline or the registry is not available (e.g. airplane, train, bad internet connection, company vpn required, etc.)
When does this actually happen? Tagged docker releases shouldn't need an update IMHO, e.g. jdk:1.8.4. And does latest have special semantics or do you need to pull the latest version manually as well?
How fast is the pull update check?

My concern is that we break the build for developers in scenarios where docker can't check for an update or the internet connection is so slow that building an image takes way longer due to this check.

Can you elaborate more on which kind of environment this is required? For sure we can put a small remark in the documentation :smile:

wsargent commented 6 years ago

So the immediate requirement -- the company I'm consulting for uses an alpine-zulu java image based off https://hub.docker.com/r/azul/zulu-openjdk-alpine/~/dockerfile/ -- the base image was put into Artifactory.

However, there was no JAVA_HOME environment variable. The base image was updated and pushed to artifactory, but none of the clients would pick up the updated version, because there were local instances of this image tagged "latest" on the local clients -- even though they were out of date.

So the only way to see if a tag is out of date is to add "--pull" to the build option, whether it's "latest" or not. There are no special semantics.

The update check is something like 0.5 milliseconds. It checks the signature hash.

My concern is that we break the build for developers in scenarios where docker can't check for an update or the internet connection is so slow that building an image takes way longer due to this check.

For that case, the error is easy:

pull access denied for <url>/alpine-zulu, repository does not exist or may require 'docker login'

You either log in, or you change the dockerBaseImage so it's pointing locally. It's fail fast, either way.

muuki88 commented 6 years ago

Sorry for the late reply.

This sounds reasonable to me. Docker is moving very fast (compared to the other build tools) so I'm always cautious on what to add what maybe removed/changed in the near future.

If you have the time you can open a pull request :)

wsargent commented 6 years ago

Adding to this -- in some cases there's an utterly bizarre issue with Docker that is exposed as a result of adding --pull https://github.com/moby/moby/issues/33252#issuecomment-363246100

This is an issue with Docker credentials specifically, not sbt, so this change is still recommended, but worth noting for posterity.

mrubin commented 5 years ago

This has been open for almost a year and a half. Are there any updates on addressing this issue? Thank you.

muuki88 commented 5 years ago

@mrubin you can open a pull request add this :smiley:

The dockerBuildOptions in the DockerPlugin can be extended.

This option should be easily revertable by this sbt setting

dockerBuildOptions -= "--pull"

sbt / sbt-native-packager