Testing different linux distributions

[heiderich]

@sebasguts suggested to run Oscar tests in different distributions, in particular in Ubuntu and ArchLinux.

Currently Jenkins runs (can run) in a docker container and the actual tests run within that same docker container. In order to test different distributions, I see at least two options:

1) install Jenkins itself in containers with different distributions and then run the tests as we currently do 2) have Jenkins start Docker containers in which the actual tests run (one for very distribution we want to test).

Option 1) may would probably require us to maintain a separate Dockerfile for each distribution. It also has the disadvantage of the test containers not being "clean" (we have Jenkins inside).

Option 2) would have the advantage of having the tests run in a "clean" environment. The naive idea would be to run "test Docker containers" inside the "Jenkins Docker container". However "Docker-in-Docker" seems to be discouraged. The alternative here would be "docker siblings". For details see for instance

https://jpetazzo.github.io/2015/09/03/do-not-use-docker-in-docker-for-ci/

Currently I think that options 2 with "docker siblings" is the best approach of the ones I mentioned.

I would like to discuss possible approaches here.

[rbehrends]

That would be nice to support; however, I think the more pressing issue would be to permit distributed builds, especially for macOS and Windows. Using a different Linux distributions can also give us additional insights into our configuration needs, but not as much as running the tests on an entirely different OS.

[heiderich]

I agree. I was mentioning it for the record and for us to keep these possible setups in mind when developing test infrastructure. Maybe we can do it right away in such a way such that it can be easily "containerized" later on.