experimental buildx.yml

ImreSamu commented 2 years ago

PROPOSAL ( updated at 2022-10-16 )

new workflow github/workflows/buildx.yml

for using a docker buildx ( qemu emulation ) for the non-AMD64 platforms
AMD64 is staying on the matured github/workflows/main.yml workflow.
minimal change in the template and the testing code.

The proposed `buildx.yml` matrix :

Experimental linux/arm64 images

buildx/qemu based images
the arm64 - alpine version tested with JIT=OFF ( BUILDX/QEMU7.0 problem ! )
generated from the default Dockerfile just added an -experimental tag postfix!

Platform	DockerHub image	PG	PostGIS	OS
`linux/arm64`	`postgis/postgis:10-3.2-experimental`	10	3.2	debian:bullseye
`linux/arm64`	`postgis/postgis:11-3.3-experimental`	11	3.3	debian:bullseye
`linux/arm64`	`postgis/postgis:12-3.3-experimental`	12	3.3	debian:bullseye
`linux/arm64`	`postgis/postgis:13-3.3-experimental`	13	3.3	debian:bullseye
`linux/arm64`	`postgis/postgis:14-3.3-experimental`	14	3.3	debian:bullseye
`linux/arm64`	`postgis/postgis:15-3.3-experimental`	15	3.3	debian:bullseye
`linux/arm64`	`postgis/postgis:15-3.3-alpine-experimental`	15	3.3	alpine:3.16

Other Experimental images

buildx/qemu based images
the ppc64le - alpine version tested with JIT=OFF ( BUILDX/QEMU7.0 problem ! )
generated from the default Dockerfile just added an -experimental tag postfix!

Platform	DockerHub image	PG	PostGIS	OS
`linux/arm/v7`	`postgis/postgis:15-3.3-alpine-experimental`	15	3.3	alpine:3.16
`linux/arm/v6`	`postgis/postgis:15-3.3-alpine-experimental`	15	3.3	alpine:3.16
`linux/386`	`postgis/postgis:15-3.3-alpine-experimental`	15	3.3	alpine:3.16
`linux/ppc64le`	`postgis/postgis:15-3.3-alpine-experimental`	15	3.3	alpine:3.16
`linux/ppc64le`	`postgis/postgis:15-3.3-experimental`	15	3.3	debian:bullseye

minimal post-tests:

postgres-basics - https://github.com/docker-library/official-images/tree/master/test/tests/postgres-basics
- used environment variables: $POSTGRES_TEST_TRIES; $POSTGRES_TEST_SLEEP
postgres-initdb - https://github.com/docker-library/official-images/tree/master/test/tests/postgres-initdb
- used environment variables: $POSTGRES_TEST_TRIES; $POSTGRES_TEST_SLEEP
postgis-basics - https://github.com/postgis/docker-postgis/tree/master/test/tests/postgis-basics ( patched )
- patched .. added environment variables: $POSTGRES_TEST_TRIES; $POSTGRES_TEST_SLEEP

The default $POSTGRES_TEST_TRIES; $POSTGRES_TEST_SLEEP is not enough for the emulation so I added a higher value for the buildx.yml

          POSTGRES_TEST_TRIES: 42
          POSTGRES_TEST_SLEEP: 2

Lesson learned / improvements:

under the QEMU7.0 ( ~emulation) the regress testing sometimes failed ( "aarch64"; "ppc64le" )
- example: LOG: server process (PID 28990) was terminated by signal 7: Bus error
- setting the JIT=OFF solved the problems in the alpine regression test. ( so added to workaround to the alpine.template )
we can use the current Makefile and make test
- with setting the correct DOCKER_DEFAULT_PLATFORM= , PLATFORM= environment variables!
  - see github/workflows/buildx.yml
the uname -a ; uname -m ; lscpu ; cat /proc/cpuinfo - useful for the buildx/qemu debugging.

NOW: using different tagging for the "buildx build" !

-experimental tag postfix has been added to the image names in the buildx.yml!
- like 15-3.3-experimental / 15-3.3-alpine-experimental

NEW environment variables: ( Makefile, workflow )

WORKFLOW: main,buildx ( workflow id )
TAG_POSTFIX: <>,'-experimental' ( Docker image tag postfix )

IMPORTANT:

I have minimal knowledge of the Makefile syntax; so please double-check!
no problem if some part of my implementation is not accepted/refactored.
docker push not tested!

ImreSamu commented 2 years ago

imho review ready ( cc @phillipross )

ImreSamu commented 2 years ago

A lot of changes have been made( Postgis 3.3; Postgres 15 Beta4 ) and will have to be implemented here. ( so the current PR is not yet ready )

phillipross commented 2 years ago

I agree that a separate tagging scheme should be used for images produced by this alternate workflow for the time being. It looks like there's still a bunch of things to figure out, but this is a great start!

If it's possible to still have CI runs occur for PRs in draft mode, maybe you can switch this PR to draft mode so that it doesn't get accidentally merged?

ImreSamu commented 2 years ago

PR converted to "draft"

I agree that a separate tagging scheme should be used for images produced by this alternate workflow for the time being.

ok . . I will add an experimental postfix .. ( example 14-3.3-experimental ) and I am open to any other suggestions.

ImreSamu commented 1 year ago

A lot of improvements / changes :

the PR first post has been updated with the relevant changes, and the proposed matrix is included.
ready to review ( merge ) ( cc @phillipross )

important: the docker push part is not tested!

ImreSamu commented 1 year ago

Added some new ENV vars + other small improvements;

example;

 env:
   ALPINE_VERSION: 3.16
   WORKFLOW: buildx
   REPO_NAME: postgis
   IMAGE_NAME: postgis
   VERSION: 15-3.3
   VARIANT: alpine
   TAG_POSTFIX: -experimental

Ready to review ...

postgis / docker-postgis