search

coala / docker-coala-base

coala base docker image
21 stars 28 forks source link

Switch to openSUSE Leap #144

Open jayvdb opened 7 years ago

jayvdb commented 7 years ago

Leap is more stable than Tumbleweed. Current problem with Tumbleweed is https://github.com/coala/docker-coala-base/issues/142 .

jayvdb commented 7 years ago

One problem encountered with 42.2 and 42.1 is gotype fails. Notified upstream. Possibly waiting on https://github.com/openSUSE/brp-check-suse/pull/7 .

jayvdb commented 7 years ago

There is also a problem with Julia https://bugzilla.opensuse.org/show_bug.cgi?id=1030975

ronisbr commented 7 years ago

No, julia is working fine in Leap 42.2 and 42.1. I just tested in 4 different machines.

I did not analyze it deeply, but it seems that you are using in this Dockerfile the Tumbleweed science repository with Leap. This is not supported and will not work.

Due to some SLE libraries, it was not possible to update julia to v0.5 in Leap 42.2, and it will not be possible to update it either in 42.3. Hence, if you want to use Leap, you will have to stick with julia v0.4, which are in the main repositories.

ronisbr commented 7 years ago

I think I found the issue. Inside the container, zypper is not installing a needed dependency of julia (maybe it is some problem with rpm not figuring out the dependencies by itself). I will look for this later (maybe I need to manually add those libraries as runtime dependencies).

In the mean time, just install the following packages:

libcholmod-3_0_6 libmpfr4 suitesparse-devel

Then, you should have access to julia v0.4.7 in Leap inside a Docker container.

jayvdb commented 7 years ago

The main versions from the 0.11 release latest build are

The next release (0.12) can have higher versions, but switching to Leap shouldnt result in lower versions than those above from 0.11.

fneu commented 7 years ago

@jayvdb Are you sure this switch is a good idea in general? Looking at #146 I'm not too sure.

The reason we went with Tumbleweed at the time was the availability of up-to-date packages in the official repositories. Main contenders at the time were arch and ubuntu. Both couldn't satisfy our package needs without the inclusion of arch's AUR or ubuntu's PPAs. I did not want to go that way because of the unsupported nature of those sources. Tumbleweed allowed us to get the up-to-date versions we needed from official repositories for the most part.

Now, I don't know about the exact nature of the problems you face with tumbleweed and I'm sure they are justyfied, but I don't see how switching to leap like this will solve them:

       # flawfinder
       --plus-repo http://download.opensuse.org/repositories/home:illuusio/openSUSE_Tumbleweed/ \
+      # php7-imagemagick
+      --plus-repo http://download.opensuse.org/repositories/home:flacco:rtk:php7/openSUSE_Leap_42.2/ \

While Leap is a more stable distribution than tumbleweed (stable as in "moves slower", not necessarily "breaks less"), coala will need to obtain a lot of dependencies from unofficial repositories that move and break equally as fast as tumbleweed. While being unofficial and therefore subject to less testing and potentially less initiative to fix things.

--

This is not a rant or anything, I'm positively interested in why you think this move will help us. I'm also wondering whether -- if we need unofficial sources anyway -- other distributions could fit the bill better in this situation. While I use openSUSE myself, debian-land seems to be be better integrated with docker, i.e.

ronisbr commented 7 years ago

Hi guys,

Leap has some very old packages because they are gathered from SLES. Given that, I was not able to update julia to version 0.5, since we can't update those packages. If you are planning to use julia > 0.4.7, then I highly suggest to use Tumbleweed. In Tumbleweed, we officially support julia 0.6 and some tests are done periodically. If you use Leap, you will have to download many dependencies from external repositories leading to a "Frankenstein" environment that probably nobody has tested.

Conclusion: sometimes things will break in Tumbleweed, much more often than in Leap. But, if you need many external repositories, then Tumbleweed will be more stable.

jayvdb commented 7 years ago

Tumbleweed breaks almost daily. Today it is https://github.com/coala/docker-coala-base/issues/216

jayvdb commented 7 years ago

There is also a problem with lua53 at the moment, but it affects Leap also. https://github.com/coala/docker-coala-base/issues/211 / https://gitlab.com/coala/package_manager/issues/156 I havent raised that one upstream yet.

jayvdb commented 7 years ago

@fneu , to be very blunt, Tumbleweed is a very rough ride, especially when we need to use packages in devel:... or worse the OBS Factory, which is the current situation. These packages are left broken for up to a week without any obvious concern from upstream maintainers that the packages are broken. Take the time to read the closed issues over the last few months. Someone has to do that work, and currently the only person doing it is me.

We already have a "Frankenstein" environment based on Tumbleweed. But luckily, we have a very solid integration test suite used to reject builds that are broken.

When we can't workaround an upstream issue quickly, additional upstream breakages occurs in the interrim, meaning the PR to get master working again requires multiple fixes, which increases the breadth of knowledge needed for one person to get a PR together, and reducing the pool of potential people who can keep this docker green.

Building custom rpms , and a custom repo in OBS, is the next step. Over the last week, I have been doing this at https://build.opensuse.org/project/show/home:jayvdb:coala , with Yuki also creating one of the RPMs. This will add stability, and opens a new pathway for adding fixes, but it will also reduce the image size substantially. We will also submit linter .rpm/.spec 's upstream into Tumbleweed, which will eventually filter back down and we can remove our custom packages. Everyone wins.

More importantly, please see https://github.com/coala/docker-coala-base/issues/203 . The current design of this repo means creating a new docker with a coala bugfix 'patch' introduces all sorts of unexpected and unrelated changes because Tumbleweed changes so quickly. That is completely unacceptable for a CI system. New releases of linters should only appear in minor releases of this coala docker, so that our users can plan for those surprises when they switch from one version of the docker image to the next (0.10 to 0.11).

... other distributions could fit the bill better in this situation

That discussion belongs elsewhere. We can have multiple docker images, with different branches/tags or even different repo/image names.

While on offtopic, IMO the biggest problem a 'mega' docker image faces is not how current the linters are, but the size of the image. I currently don't use this image where I should, due to the size of it (and thus the legnth of time it takes to fetch the image in a CI worker). Instead I hand roll my own, with custom scripts. And we've tried reducing the size of the image using other distros; the results are worse, or similar size.

jayvdb commented 7 years ago

Note I've tested switching to Leap 42.3, and it works well, but the highest base image is 42.2 despite https://github.com/openSUSE/docker-containers/issues/55 being closed as fixed.

jayvdb commented 7 years ago

https://hub.docker.com/_/opensuse/ now has 'leap' pointing at 42.3, and 42.2 is the old one.

jayvdb commented 7 years ago

Note an update to https://github.com/coala/docker-coala-base/issues/144#issuecomment-315105919

The most recent docker build of 0.11 release (unintentionally) upgraded Julia from 0.5.1 to 0.6.0

Leap 42.3 currently has