Building multiarch catalogs

didierofrivia commented 3 weeks ago

This PR aims to solve the issue of building catalogs for different architectures using just buildah, as reported in https://github.com/Kuadrant/authorino-operator/pull/214#issuecomment-2373038118 . It uses a matrix strategy in order to assert the right OPM base image is set in the catalog Dockerfile and build accordingly. It accomplish this having 2 different jobs, the original one is kept, expanded to the other 2 missing archs, and a dedicated one was created to explicitly include the opm version and architecture desired. The later one will create additional catalog images of the form authorino-operator-catalog:{tag}-{arch}

Notes:

Manifests generated from the single multi platform catalog build:

⇒  docker manifest inspect quay.io/kuadrant/authorino-operator-catalog:building-multiarch-catalogs
{
   "schemaVersion": 2,
   "mediaType": "application/vnd.docker.distribution.manifest.list.v2+json",
   "manifests": [
      {
         "mediaType": "application/vnd.docker.distribution.manifest.v2+json",
         "size": 2846,
         "digest": "sha256:6e5571ece6060c8e0830389838be8944cc2da442b271c7e5a63cb3674f17ca52",
         "platform": {
            "architecture": "amd64",
            "os": "linux"
         }
      },
      {
         "mediaType": "application/vnd.docker.distribution.manifest.v2+json",
         "size": 2846,
         "digest": "sha256:d812e9323dc66d0e9afe65f46d12e549b63de9b72b9d27bd481bdc69b3dfc6ea",
         "platform": {
            "architecture": "arm64",
            "os": "linux"
         }
      },
      {
         "mediaType": "application/vnd.docker.distribution.manifest.v2+json",
         "size": 2846,
         "digest": "sha256:ec43f6999520f0e79593f67199d76ac1251c7f5c94a6ad5a8e8c8e944f7c0ad6",
         "platform": {
            "architecture": "s390x",
            "os": "linux"
         }
      },
      {
         "mediaType": "application/vnd.docker.distribution.manifest.v2+json",
         "size": 2846,
         "digest": "sha256:9f07f4a80c237c00bb60f9ce38111bc421219b4f6f5be0a00f7403fdaa8681a3",
         "platform": {
            "architecture": "ppc64le",
            "os": "linux"
         }
      }
   ]
}

It was generated with the latest tag of opm. When pulling an specific platform you get:

⇒  docker pull quay.io/kuadrant/authorino-operator-catalog:building-multiarch-catalogs --platform linux/ppc64le
...
 ⇒  docker image inspect quay.io/kuadrant/authorino-operator-catalog:building-multiarch-catalogs
[
    {
        "Id": "sha256:e76120cba703865877c62f1377f707ea22a8f83fc8973d660795e501dedd9140",
        "RepoTags": [
            "quay.io/kuadrant/authorino-operator-catalog:building-multiarch-catalogs"
        ],
        "RepoDigests": [
            "quay.io/kuadrant/authorino-operator-catalog@sha256:fe358575f2df27d6d7973027aa4a47571238f6a86c900370dea075f04a712390"
        ],
        "Parent": "",
        "Comment": "FROM quay.io/operator-framework/opm:latest",
        "Created": "2024-11-06T19:16:22.919202023Z",
        "DockerVersion": "",
        "Author": "",
        "Config": {
            "Hostname": "1d4f59372fd6",
            "Domainname": "",
            "User": "1001",
            "AttachStdin": false,
            "AttachStdout": false,
            "AttachStderr": false,
            "Tty": false,
            "OpenStdin": false,
            "StdinOnce": false,
            "Env": [
                "PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/busybox",
                "SSL_CERT_FILE=/etc/ssl/certs/ca-certificates.crt"
            ],
            "Cmd": [
                "serve",
                "/configs",
                "--cache-dir=/tmp/cache"
            ],
            "Image": "",
            "Volumes": {},
            "WorkingDir": "/",
            "Entrypoint": [
                "/bin/opm"
            ],
            "OnBuild": [],
            "Labels": {
                "io.buildah.version": "1.22.3",
                "operators.operatorframework.io.index.configs.v1": "/configs"
            }
        },
        "Architecture": "ppc64le",
        "Os": "linux",
        "Size": 78990744,
        "GraphDriver": {
            "Data": {
                "LowerDir": "/var/lib/docker/overlay2/c7ca26cffc25e28bd032c502d8be0008d45590f0ae983452872ceb1f9908f143/diff:/var/lib/docker/overlay2/9a3aad3a405cd34a349867d95293a7c2f8dc066a4dd5132211136585ecc9fee9/diff:/var/lib/docker/overlay2/f980a24e5be7e03572d719d869c10440087ba5de2b602f2cbe6c9a44f9976743/diff:/var/lib/docker/overlay2/f71ad2363963c328141f8987746ef980ac4e3d7ca15aebf2107d0c4591df6eec/diff:/var/lib/docker/overlay2/06074fe4a1f93c8f8674b66ca8dce3b78d60bb5666cef34562196b54323ca471/diff:/var/lib/docker/overlay2/495d82ba6ba26ea024dbe0bc48de2302e0721e8777a9744be52ab9bb37e49e00/diff:/var/lib/docker/overlay2/1f004cb15f3412f6e62737371b7d043a6228bfbb1a43df463e479b41b58cc794/diff:/var/lib/docker/overlay2/bd9d1a77886a471f959b8a07db8a10906503f244c0bafc3fe709c7e958595e8a/diff:/var/lib/docker/overlay2/84190492b362337bca2aca2f2cf0884d256c05a7d6b1d0724b1f8508ae44323b/diff:/var/lib/docker/overlay2/b2e73f949e34583390ae4ed477e7149e5ddd4f7f99bb369f5ebda99638f1525f/diff:/var/lib/docker/overlay2/b01bb0de0b69d1d12c9b55e3db011b96f9b9fdada0d830223b58db456c0c55de/diff:/var/lib/docker/overlay2/3e1df8ca285fff7a522840dc17ecd9b28df8280ecb3ab296d29ce97317990b45/diff:/var/lib/docker/overlay2/9bf108f6ee724b9d5fa4da5526c43232979dac2070834d0b42f727b4c6b977f0/diff:/var/lib/docker/overlay2/fdf01a45ce06fbb5020b2679aa9d4ae50ec4d90901b1678618c86214522d39e7/diff:/var/lib/docker/overlay2/b514bbc5f78da4e54dad71d6eeab90d5a1ac569aab2da848dab3e7b9820cbfcf/diff",
                "MergedDir": "/var/lib/docker/overlay2/ae48a70d547c3f790c412b4b4701fdc843d8db8c80e24685d9e2420ae0d0bb5f/merged",
                "UpperDir": "/var/lib/docker/overlay2/ae48a70d547c3f790c412b4b4701fdc843d8db8c80e24685d9e2420ae0d0bb5f/diff",
                "WorkDir": "/var/lib/docker/overlay2/ae48a70d547c3f790c412b4b4701fdc843d8db8c80e24685d9e2420ae0d0bb5f/work"
            },
            "Name": "overlay2"
        },
        "RootFS": {
            "Type": "layers",
            "Layers": [
                "sha256:4dc35ec3c9cd6c8fa67419fddfda7d10a17e4b7156a7a7c2e226e4e5b2470a7d",
                "sha256:8fa10c0194df9b7c054c90dbe482585f768a54428fc90a5b78a0066a123b1bba",
                "sha256:ddc6e550070ca022d94bd4415de20545ba69954033b4985045a8b05f538bbe5c",
                "sha256:4d049f83d9cf21d1f5cc0e11deaf36df02790d0e60c1a3829538fb4b61685368",
                "sha256:af5aa97ebe6ce1604747ec1e21af7136ded391bcabe4acef882e718a87c86bcc",
                "sha256:ac805962e47900b616b2f4b4584a34ac7b07d64ac1fd2c077478cf65311addcc",
                "sha256:bbb6cacb8c82e4da4e8143e03351e939eab5e21ce0ef333c42e637af86c5217b",
                "sha256:2a92d6ac9e4fcc274d5168b217ca4458a9fec6f094ead68d99c77073f08caac1",
                "sha256:1a73b54f556b477f0a8b939d13c504a3b4f4db71f7a09c63afbc10acb3de5849",
                "sha256:f4aee9e53c42a22ed82451218c3ea03d1eea8d6ca8fbe8eb4e950304ba8a8bb3",
                "sha256:b336e209998fa5cf0eec3dabf93a21194198a35f4f75612d8da03693f8c30217",
                "sha256:85f5c24580cdd27f13ddda37afa51db060f4ad25ef7d2e6b8207780c5daa008e",
                "sha256:684a549d0840e3bb5ba2beae3579802a8f28231937c9e231fe93ed204dcba327",
                "sha256:8b8e65cdc40ae490eb20e053ff31bd9bd39c836f9b0f26f65bb6b4d9dc120400",
                "sha256:b3196e830b67702ca1a8d024fff6379d82adb9131aef7b6a2dc66af6c9674685",
                "sha256:da61013bb62a9a01b946b975a91fcecb3c8430b1cbd8f61be01f5030ed780f56"
            ]
        },
        "Metadata": {
            "LastTagTime": "0001-01-01T00:00:00Z"
        }
    }
]

No manifests generated since it's a single build when you inspect the ones from the specific multi arch job, just its layers:

⇒  docker manifest inspect quay.io/kuadrant/authorino-operator-catalog:building-multiarch-catalogs-ppc64le
{
"schemaVersion": 2,
"mediaType": "application/vnd.docker.distribution.manifest.v2+json",
"config": {
    "mediaType": "application/vnd.docker.container.image.v1+json",
    "size": 4549,
    "digest": "sha256:6be66ab24ce6cdcab5b4faea287ab99cde7b504b125790ec43e845da9e3571d7"
},
"layers": [
    {
        "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip",
        "size": 84606,
        "digest": "sha256:4d4401f0320bd6f39c22d9a4a0eba68686c97d1928363283fc47ba8a8dee6382"
    },
    {
        "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip",
        "size": 12579,
        "digest": "sha256:2e4cf50eeb92ac3a7afe75e15d96a26dee99449f86b46c75b5d95f4418a5bca0"
    },
    {
        "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip",
        "size": 452510,
        "digest": "sha256:6f4cfee9177b9f884e8d86b48261a25094b2fcea1a7920919f47ea00712dbee8"
    },
    {
        "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip",
        "size": 75,
        "digest": "sha256:0f8b424aa0b96c1c388a5fd4d90735604459256336853082afb61733438872b5"
    },
    {
        "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip",
        "size": 193,
        "digest": "sha256:d557676654e572af3e3173c90e7874644207fda32cd87e9d3d66b5d7b98a7b21"
    },
    {
        "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip",
        "size": 130,
        "digest": "sha256:c8022d07192eddbb2a548ba83be5e412f7ba863bbba158d133c9653bb8a47768"
    },
    {
        "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip",
        "size": 173,
        "digest": "sha256:d858cbc252ade14879807ff8dbc3043a26bbdb92087da98cda831ee040b172b3"
    },
    {
        "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip",
        "size": 97,
        "digest": "sha256:1069fc2daed1aceff7232f4b8ab21200dd3d8b04f61be9da86977a34a105dfdc"
    },
    {
        "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip",
        "size": 382,
        "digest": "sha256:b40161cd83fc5d470d6abe50e87aa288481b6b89137012881d74187cfbf9f502"
    },
    {
        "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip",
        "size": 326,
        "digest": "sha256:3f4e2c5863480125882d92060440a5250766bce764fee10acdbac18c872e4dc7"
    },
    {
        "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip",
        "size": 129107,
        "digest": "sha256:80a8c047508ae5cd6a591060fc43422cb8e3aea1bd908d913e8f0146e2297fea"
    },
    {
        "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip",
        "size": 834946,
        "digest": "sha256:c57d5d2ad6083e98b301e2cb9283489173321ca77397fb6c150bfda946173fdb"
    },
    {
        "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip",
        "size": 4026819,
        "digest": "sha256:ba63ca86b039286f74b2529ac458da1de06035b23bd9b74f2fae2484c30700c0"
    },
    {
        "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip",
        "size": 157,
        "digest": "sha256:c072e97f89853830431248238603055ecfb103c6ad3386e3ff8fd46fc9beace6"
    },
    {
        "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip",
        "size": 17903774,
        "digest": "sha256:31b3e74066eb1f3153fceede0b1299ee0492c1573dcf5d6a8181b12b5b3bc788"
    },
    {
        "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip",
        "size": 74900,
        "digest": "sha256:21311f22599992b52d76a7fd5105e6f78e8e4b7ecb300c48aad921bfb885e8b5"
    }
]
}

It was generated with the v1.48.0-{ARCH} tag of opm When pulling a catalog image from the specific multi arch job:

⇒  docker pull quay.io/kuadrant/authorino-operator-catalog:building-multiarch-catalogs-ppc64le
...
⇒  docker image inspect quay.io/kuadrant/authorino-operator-catalog:building-multiarch-catalogs-ppc64le
[
    {
        "Id": "sha256:6be66ab24ce6cdcab5b4faea287ab99cde7b504b125790ec43e845da9e3571d7",
        "RepoTags": [
            "quay.io/kuadrant/authorino-operator-catalog:building-multiarch-catalogs-ppc64le"
        ],
        "RepoDigests": [
            "quay.io/kuadrant/authorino-operator-catalog@sha256:47ae5ba91227f0df010904fb258edb94603b482c62ac0456009b8d1a5ef00866"
        ],
        "Parent": "",
        "Comment": "FROM quay.io/operator-framework/opm:v1.48.0-ppc64le",
        "Created": "2024-11-06T19:37:48.315754425Z",
        "DockerVersion": "",
        "Author": "",
        "Config": {
            "Hostname": "2ee92a32997c",
            "Domainname": "",
            "User": "1001",
            "AttachStdin": false,
            "AttachStdout": false,
            "AttachStderr": false,
            "Tty": false,
            "OpenStdin": false,
            "StdinOnce": false,
            "Env": [
                "PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/busybox",
                "SSL_CERT_FILE=/etc/ssl/certs/ca-certificates.crt"
            ],
            "Cmd": [
                "serve",
                "/configs",
                "--cache-dir=/tmp/cache"
            ],
            "Image": "",
            "Volumes": {},
            "WorkingDir": "/",
            "Entrypoint": [
                "/bin/opm"
            ],
            "OnBuild": [],
            "Labels": {
                "io.buildah.version": "1.22.3",
                "operators.operatorframework.io.index.configs.v1": "/configs"
            }
        },
        "Architecture": "ppc64le",
        "Os": "linux",
        "Size": 78990744,
        "GraphDriver": {
            "Data": {
                "LowerDir": "/var/lib/docker/overlay2/c7ca26cffc25e28bd032c502d8be0008d45590f0ae983452872ceb1f9908f143/diff:/var/lib/docker/overlay2/9a3aad3a405cd34a349867d95293a7c2f8dc066a4dd5132211136585ecc9fee9/diff:/var/lib/docker/overlay2/f980a24e5be7e03572d719d869c10440087ba5de2b602f2cbe6c9a44f9976743/diff:/var/lib/docker/overlay2/f71ad2363963c328141f8987746ef980ac4e3d7ca15aebf2107d0c4591df6eec/diff:/var/lib/docker/overlay2/06074fe4a1f93c8f8674b66ca8dce3b78d60bb5666cef34562196b54323ca471/diff:/var/lib/docker/overlay2/495d82ba6ba26ea024dbe0bc48de2302e0721e8777a9744be52ab9bb37e49e00/diff:/var/lib/docker/overlay2/1f004cb15f3412f6e62737371b7d043a6228bfbb1a43df463e479b41b58cc794/diff:/var/lib/docker/overlay2/bd9d1a77886a471f959b8a07db8a10906503f244c0bafc3fe709c7e958595e8a/diff:/var/lib/docker/overlay2/84190492b362337bca2aca2f2cf0884d256c05a7d6b1d0724b1f8508ae44323b/diff:/var/lib/docker/overlay2/b2e73f949e34583390ae4ed477e7149e5ddd4f7f99bb369f5ebda99638f1525f/diff:/var/lib/docker/overlay2/b01bb0de0b69d1d12c9b55e3db011b96f9b9fdada0d830223b58db456c0c55de/diff:/var/lib/docker/overlay2/3e1df8ca285fff7a522840dc17ecd9b28df8280ecb3ab296d29ce97317990b45/diff:/var/lib/docker/overlay2/9bf108f6ee724b9d5fa4da5526c43232979dac2070834d0b42f727b4c6b977f0/diff:/var/lib/docker/overlay2/fdf01a45ce06fbb5020b2679aa9d4ae50ec4d90901b1678618c86214522d39e7/diff:/var/lib/docker/overlay2/b514bbc5f78da4e54dad71d6eeab90d5a1ac569aab2da848dab3e7b9820cbfcf/diff",
                "MergedDir": "/var/lib/docker/overlay2/e4a693c34a4a6820df8c5a1756f6999cc7728a3f97aff33f4e1c193ef0828d77/merged",
                "UpperDir": "/var/lib/docker/overlay2/e4a693c34a4a6820df8c5a1756f6999cc7728a3f97aff33f4e1c193ef0828d77/diff",
                "WorkDir": "/var/lib/docker/overlay2/e4a693c34a4a6820df8c5a1756f6999cc7728a3f97aff33f4e1c193ef0828d77/work"
            },
            "Name": "overlay2"
        },
        "RootFS": {
            "Type": "layers",
            "Layers": [
                "sha256:4dc35ec3c9cd6c8fa67419fddfda7d10a17e4b7156a7a7c2e226e4e5b2470a7d",
                "sha256:8fa10c0194df9b7c054c90dbe482585f768a54428fc90a5b78a0066a123b1bba",
                "sha256:ddc6e550070ca022d94bd4415de20545ba69954033b4985045a8b05f538bbe5c",
                "sha256:4d049f83d9cf21d1f5cc0e11deaf36df02790d0e60c1a3829538fb4b61685368",
                "sha256:af5aa97ebe6ce1604747ec1e21af7136ded391bcabe4acef882e718a87c86bcc",
                "sha256:ac805962e47900b616b2f4b4584a34ac7b07d64ac1fd2c077478cf65311addcc",
                "sha256:bbb6cacb8c82e4da4e8143e03351e939eab5e21ce0ef333c42e637af86c5217b",
                "sha256:2a92d6ac9e4fcc274d5168b217ca4458a9fec6f094ead68d99c77073f08caac1",
                "sha256:1a73b54f556b477f0a8b939d13c504a3b4f4db71f7a09c63afbc10acb3de5849",
                "sha256:f4aee9e53c42a22ed82451218c3ea03d1eea8d6ca8fbe8eb4e950304ba8a8bb3",
                "sha256:b336e209998fa5cf0eec3dabf93a21194198a35f4f75612d8da03693f8c30217",
                "sha256:85f5c24580cdd27f13ddda37afa51db060f4ad25ef7d2e6b8207780c5daa008e",
                "sha256:684a549d0840e3bb5ba2beae3579802a8f28231937c9e231fe93ed204dcba327",
                "sha256:8b8e65cdc40ae490eb20e053ff31bd9bd39c836f9b0f26f65bb6b4d9dc120400",
                "sha256:b3196e830b67702ca1a8d024fff6379d82adb9131aef7b6a2dc66af6c9674685",
                "sha256:04ad95672c685937247a7d9e2827d6c404f51bf9482832d57d327c55ca8cf626"
            ]
        },
        "Metadata": {
            "LastTagTime": "0001-01-01T00:00:00Z"
        }
    }
]

The original (now modified) job should work, since buildah is using the latest tag that it's built multi platform too https://quay.io/repository/operator-framework/opm?tab=tags&tag=latest... but apparently it doesn't (?) hence the explicit multi arch job. @R3hankhan123 needs to validate this.
Other attempts made:
- https://github.com/Kuadrant/authorino-operator/pull/225 - using uraimo/run-on-arch-action@v2
- https://github.com/Kuadrant/authorino-operator/pull/226 - passing args to the Dockerfile at buildah build time
The issue was that the dockerfile was generated by a x86 builder, instead of a multiarch one

codecov-commenter commented 3 weeks ago

:warning: Please install the 'codecov app svg image' to ensure uploads and comments are reliably processed by Codecov.

Codecov Report

All modified and coverable lines are covered by tests :white_check_mark:

Project coverage is 61.78%. Comparing base (05fddaa) to head (ba8571b).

Additional details and impacted files

```diff @@ Coverage Diff @@ ## main #222 +/- ## ======================================= Coverage 61.78% 61.78% ======================================= Files 2 2 Lines 785 785 ======================================= Hits 485 485 Misses 249 249 Partials 51 51 ``` | [Flag](https://app.codecov.io/gh/Kuadrant/authorino-operator/pull/222/flags?src=pr&el=flags&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Kuadrant) | Coverage Δ | | |---|---|---| | [unit](https://app.codecov.io/gh/Kuadrant/authorino-operator/pull/222/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Kuadrant) | `61.78% <ø> (ø)` | | Flags with carried forward coverage won't be shown. [Click here](https://docs.codecov.io/docs/carryforward-flags?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Kuadrant#carryforward-flags-in-the-pull-request-comment) to find out more.

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.

R3hankhan123 commented 2 weeks ago

Hi @didierofrivia looks like after the modifications made to the catalog.mk file the matrix approach is no longer required as its working fine on s390x and ppc64le. Cheers

Kuadrant / authorino-operator

Building multiarch catalogs #222

Codecov Report