Provide CI/CD-Environment for multiple docker versions

[leipert]

Discussion directly coming from here: https://github.com/nlf/dlite/issues/61

To be future-proof, a build and test environment should be created, that is able to build multiple images for each (?!) minor docker version?

My proposal would be:

• Let Travis CI build an image matrix and test it with the current version of dlite and docker

Should I try to implement such a thing with travis?

[nlf]

i'm trying to figure out what this would look like in a general sense, i feel like version numbers could get out of control quickly if we're including the docker version in them, as in 2.1.1-1.9.1 for the current release of dhyve-os which contains docker 1.9.1

[leipert]

The releases would be still like x.y.z. But you would attach multiple binaries to the release, like syncthing does. So the binaries would have names like:

• dhyve-os-docker1.8-vx.y.z
• dhyve-os-docker1.9-vx.y.z

or shorter:

• dhyve-os-d1.8-x.y.z
• dhyve-os-d1.9-x.y.z

And the naming should be no problem, as an automated process would do it :)

[leipert]

@nlf Maybe now is the time we should start talking about it: https://github.com/docker/docker/releases/tag/v1.10.0

[nlf]

yup, i think you're right.

ok - so the issue is that we have one "base" vm and it needs the ability to fetch X versions of docker. given that docker is a single binary, what do you think about removing docker from the vm and having an init script that downloads a version of docker passed in via command line?

[endersonmaia]

@nfl I think the script solutions is best.

Binary releases are available here : https://github.com/docker/docker/releases

The latest docker-1.10.0 for linux direct download : https://get.docker.com/builds/Linux/x86_64/docker-1.10.0

The script could check the GitHub API via curl it there's a new release compared to the installed release, and do the upgrade. And won't be necessary to upgrade the OS image.

[leipert]

@endersonmaia What if I want to pin down a docker version?

Would it be hard to have

• dhyve-os-docker1.9-v2.1.1
• dhyve-os-docker1.10-v2.1.1

images and let the user decide which one he will use?

I mean dlite update could also just check which docker version the user has installed and download the appropriate image?

[nlf]

this is why i was considering downloading it via init script. the vm image would have no docker installed by default.

dlite would configure the requested version of docker and store it in the config.json, defaulting to the newest release at the time the installation is run.

when starting the daemon, the docker version will get passed in the kernel command line, i.e. docker_version=1.10.0

when booting, an init script would ensure that docker is installed, and matches the version requested in the kernel command line.

this means that for updating docker, you would dlite stop edit your config.json to change the requested docker version, then dlite start. when the vm boots it will see that the previously installed version of docker doesn't match the requested version, and download the requested version automatically.

[leipert]

This sounds like a good solution to me :+1:

[endersonmaia]

:+1:

[endersonmaia]

the latest debate was on OFF-TOPIC, but this solution will enable the CI/CD matrix that @leipert suggested

[leipert]

On topic: It seems like a testing dhyve-os might not be possible, as dlite start fails on travis: https://travis-ci.org/leipert/dlite/jobs/107334093 sysctl kern.hv_support returns null. I will try it on circle CI.

[leipert]

circleCI does not let me run OS X builds. I found no other free CI service for OSX.

I will close this issue for now. If anyone has an old mac which is new enough for running dlite, we could set up a CI env there...

Regarding the init script, I just opened: #16

[endersonmaia]

dhyve-os is a minimal linux, we shouldn't need OS X to test it. I know it would be need to test the boot under xhyve hypervisor, but maybe to test internal dhyve functionality, we only need a normal qemu/KVM vm, or even VirtualBox.

what do you think ?

[nlf]

i have no idea what that testing would look like.. testing that it boots is probably pretty easy, but how would we test things like the various init scripts? how would we know what the vm running in the test is doing?

[endersonmaia]

• Check if all needed sysctl were enabled/disabled;
• Check if init script started correctly;
• Check for file permissions;
• Check for configuration;
• Check if the right version of docker was installed when passed the version (#16);

This way on any new contribution/PR, the script will check if someone didn't messed up with anything.

Now that you mentioned it, I really don't know how to get this output from outside the VM for checking the test results.

[cmosetick]

@nlf I like your proposal here. It allows the user to have maximum flexibility on their own. :thumbsup:

this is why i was considering downloading it via init script. the vm image would have no docker installed by default.

dlite would configure the requested version of docker and store it in the config.json, defaulting to the newest release at the time the installation is run.

when starting the daemon, the docker version will get passed in the kernel command line, i.e. docker_version=1.10.0

when booting, an init script would ensure that docker is installed, and matches the version requested in the kernel command line.

this means that for updating docker, you would dlite stop edit your config.json to change the requested docker version, then dlite start. when the vm boots it will see that the previously installed version of docker doesn't match the requested version, and download the requested version automatically.