search

StackStorm / st2packaging-dockerfiles

Dockerfiles used to build and test StackStorm deb/rpm packages
https://github.com/StackStorm/st2-packages
32 stars 33 forks source link

Ubuntu 20.04 support #105

Closed nzlosh closed 3 years ago

nzlosh commented 3 years ago

Add Ubuntu 20.04 support. Remove duplication of ssh keys and setup script for all builds. Remove patch for Ubuntu 13.04

nzlosh commented 3 years ago

There are unmet dependencies inside the focal build. The dh-virtualenv is being pulled from https://github.com/StackStorm/dh-virtualenv. It appears dh-virtualenv reuiqres python2, but focal doesn't have it.

dpkg-checkbuilddeps: error: Unmet build dependencies: python (>= 2.6.6-3~)
dpkg-buildpackage: warning: build dependencies/conflicts unsatisfied; aborting
dpkg-buildpackage: warning: (Use -d flag to override.)
nzlosh commented 3 years ago

I don't know why those images are failing, they build correctly on my machine.

nzlosh commented 3 years ago

@armab the PR's been cleaned up to include only focal changes.

arm4b commented 3 years ago

Just enabled focal Dockerfiles build in the Docker Hub, so the next commits to this PR should start building those images.

nzlosh commented 3 years ago

Thanks to @blag getting the patches accepted upstream :100:, I was able to switch the focal build process to dh-virtualenv upstream that supports python3.

Building the dh-virtualenv package each time we build the docker image seems redundant. Perhaps we should build the u1804 and u2004 package and then host the packages in the st2 packagecloud.io repository?

arm4b commented 3 years ago

@nzlosh Building the dedicated .deb package for dh-virtualenv and uploading it to our PackageCloud is nice, but will that require another repo and CI/CD pipeline? From the other side, this Dockerfiles repository is updated once in a half-year. I guess a few more minutes of the build time here is OK to avoid more moving pieces and level of indirection. There's already excessive complexity around the packaging.

nzlosh commented 3 years ago

Now that the patch is merged upstream, it's just a matter of time before Ubuntu and Debian packages will update their source tarballs and it will be available as part of the standard OS distribution.

We should work towards that result since it means less work for us. We can build the .deb with the patch included as a once off. We used the same dh-virtualenv for 4 years, we can reasonably expect a package to get the same life span. The other advantage is this will make Dockerfiles reproducible and less complex.

arm4b commented 3 years ago

In this situation, reproducibility and build timing is comparable.

The feedback I got from multiple engineers is that when someone jumps into the Packaging it's quite a big journey with many moving pieces involved. Having a new step in the rabbit hole: st2 -> st2-packages -> st2-dockerfiles -> dh-virtualenv -> + (new) dh-virtualenv .deb build -> new dh-virtualenv PackageCloud might bring more confusion. Instead of that, how do we go with less steps in the pipeline for folks to dig in easily, navigate and make changes?

arm4b commented 3 years ago

In the context of reproducibility, let's pin the dh-virtualenv to the latest git commit to merge this PR. I guess the real difficulty might be in the next step verifying if this latest bundled dh-virtualenv really works with the current st2-packages environment.