Release strategy and release tagging

[grantmacken]

Just to clarify. Each eXist release should have a corresponding docker tagged release version. The only arg needed for the docker build is the eXist release version The version tagged latest should be the should be the stable version not the RC version?

[adamretter]

Each eXist release should have a corresponding docker tagged release version.

Absolutely!

The only arg needed for the docker build is the eXist release version

When I did the 5.0.0-RC3 release yesterday, the process I followed was:

$ docker build --build-arg BRANCH=eXist-5.0.0-RC3 --tag existdb/existdb:5.0.0-RC3 .
$ docker push

The version tagged latest should be the should be the stable version not the RC version?

@duncdrum can correct me if I am wrong... but, as I understand it:

1. latest Docker Image: is built from our develop branch. So that is whatever is currently happening with eXist-db, right now that is 5.0.0-SNAPSHOT.

2. stable Docker Images: is built from our master branch. That will be the last stable release, at the moment that means the 4.x.x line, i.e. right now 4.3.1.

[duncdrum]

Almost:

1. :latest the development branch which currently is at 5.0.0-SNAPSHOT.
2. :release is tracking the master branch of the source-code repo, which contains stable releases, i.e. 4.3.1,

We publish updated docker images upon every commit to develop, or master. Commits to master only happen during official releases, commits to develop are ongoing. We discussed automating RC releases but decided against it, since these are an exception to the normal develop workflow of exist, and doing them manually is a simple two liner.

Cache busting changes to master only happen if either a) there is a new official release, or b) one of the core apps, e.g. exide gets updated, c) we make changes here in the docker repo. So while docker hub keeps rebuilding the master branch often, it will only result in changed images when there is a good reason for it.

My current thinking is to use github tags in an env hook, to get the version string to create the docker tags. latest and release we can set either in the hook or on dockerhub, based on the source-code branches being build

[adamretter]

@duncdrum It is very important to me that we can publish eXist-db Git tags as releases. I want to be able to have any tag that we create published as a release.

I need this so that I can compare behaviour easily between past versions of eXist-db when checking for regressions or performance changes.

As we only use Git tags for releases, I think we should have a hook which builds Git tags, generating an official point release Docker image .

[duncdrum]

@adamretter well the straight-forward way of doing this would be do keep the docker files in the main repo.

We are already publishing every commit as a release, so we are more fine-grained than tags. Since the BRANCH build arg understands git tags, we just need to add a dynamic check for the latest git tag, strip eXist- (via regEx) and tag images with the result of that operation. So we get existdb/existdb:5.0.0-RC3 instead of existdb/existdb:eXist-5.0.0-RC3

[adamretter]

@duncdrum So we can have Git tags, auto build and published?

Great, how do I get that going?

[grantmacken]

$ docker build --build-arg BRANCH=eXist-5.0.0-RC3 --tag existdb/existdb:5.0.0-RC3 .
$ docker push

The build creates the tagged image, when pushed the image is available via the dockerhub registry You can also within 'dockerhub' , get dockerhub do the building and pushing to add the image to the registry. If you set this up, when you push to this Git repo, a build/push will occur on dockerhub and not on your system.

It is a design choice, what should be built, I think, @duncdrum original intention, was to provide a away to build on each commit to the main eXist repo. I'm of the opinion, that we should only build and push only tagged github eXist releases to the dockerhub registry.

Since these eXist releases are available @ https://github.com/eXist-db/exist/archive/

ARG RELEASE=3.4.1
ENV RELEASE_ARCHIVE "https://github.com/eXist-db/exist/archive/eXist-${RELEASE}.tar.gz" 

wget/curl will get the release rather than installing git then cloning in the Dockefile.

[duncdrum]

@grantmacken actually there are a number of use cases for the :latest builds created by every commit. The two most pressing are QnA of exist-db issues, where starting an ephemeral clean exist with all the latest commits for debugging is a huge time saver. The second being CI. With :latest users can test their apps against the current stable release and upcoming changes in parallel. This way you get an early warning when changes to exist-db will break your app.

I m therefore strongly in favor to keep offering the :latest images. If we were to decide to no longer offer them and to only provide :release we can do so on the fly, by changing the webhook from commit to release in exist-db/exist that currently triggering auto-building on docker hub.

building from archived releases will save us some time during building, but means that we would need to strip a lot of the things when copying files between build stages, which are currently stripped by using the minimal build flag. Somehow, using minimal builds strikes me as the simpler solution, it also means that out of the box the apps (Dashboard, monex, etc) are always on the latest suitable version. With the archives users have to log in and update them manually after downloading. @adamretter thoughts?

[grantmacken]

I think I am approaching this from a different angle. I don't think we should expect the user to clone this repo then build the image, then run. The eXist docker images should available as release artifacts like the eXist binaries. The user story, would be something like.

• create project
• add docker-compose file
• on command line
• enter docker-compose up -d

The compose file declares what tagged container should run. The latest tag is just a tag, it can mean what we want it to mean. However I think, using the latest tag, will be the default action for most users so giving them the latest stable version, would be the best option.

[duncdrum]

@grantmacken I m confused users don t need to clone this repo. I was talking about cloning exist-db's source-code during the build phase, instead of downloading a compiled version of exist.

To use the develop branch images simply docker pull existdb/existdb:latest no interaction with this repo necessary.

[grantmacken]

building from archived releases will save us some time during building, but means that we would need to strip a lot of the things when copying files between build stages, which are currently stripped by using the minimal build flag

That is actually what happens now. Build.sh clones the eXist repo CDs into repo for then builds using build.sh ( the file in eXist ). The script then copies into only essential directories and files into a minimal directory.

[duncdrum]

Just to be clear if folks feel this strongly about latest pointing to a stable release. We can switch to using :latest in the master branch, and use :develop (?) in the develop branch. As long as we keep publishing new images with each commit, i don't care what they are called.

As for the sequence of users adding docker-compose (how? from scratch? copying it? from where?) I m afraid I still don't understand the problem. It is already possible to do just that: image: existdb/existdb:release will get you stable and image: existdb/existdb will get you develop. What am I missing?

[duncdrum]

Something Adam and I were having a look at Best Practices

[duncdrum]

about the speed and size advantages see 5ad84b1 . as for tagging images, the auto-tagging is set-up and working. Major changes at this point will need to be considered in light of the existing tags, that folks might use. So we should discuss this in a new issue.