canonical / rockcraft

Tool to create OCI Images using the language from Snapcraft and Charmcraft.
GNU General Public License v3.0
32 stars 40 forks source link

Make ROCK base image registry configurable. #655

Open aznashwan opened 1 month ago

aznashwan commented 1 month ago

What needs to get done

rockcraft.oci.Image class needs extending to add configurable registry URL during the skopeo copy of the base image to allow from pulling from non-ECR registries.

Will probably need adding a top-level rockcraft argument and transparently propagating it down to the actual image URL composed here which later gets passed to the skopeo copy command.

Why it needs to get done

Although ECR is considerably more lenient than DockeHub in terms of data pull limits for unverified requests, it still has them, leading to errors like these:

024-07-29 11:06:56.991 Execute process: ['/snap/rockcraft/1783/bin/skopeo', '--insecure-policy', '--override-arch', 'amd64', 'copy', '--retry-times', '5', 'docker://public.ecr.aws/ubuntu/ubuntu:22.04', 'oci:/root/images/ubuntu:22.04'], kwargs={}
2024-07-29 11:10:30.466 Failed to copy image: Command '['/snap/rockcraft/1783/bin/skopeo', '--insecure-policy', '--override-arch', 'amd64', 'copy', '--retry-times', '5', 'docker://public.ecr.aws/ubuntu/ubuntu:22.04', 'oci:/root/images/ubuntu:22.04']' returned non-zero exit status 1. (time="2024-07-29T11:07:28Z" level=warning msg="Failed, retrying in 1s ... (1/5). Error: initializing source docker://public.ecr.aws/ubuntu/ubuntu:22.04: reading manifest 22.04 in public.ecr.aws/ubuntu/ubuntu: toomanyrequests: Data limit exceeded"

Depending on the project being rock'd, these limits can be quite the nuisance.

The canonical/harbor-rocks repo for example defines rocks for 10 components x 3 versions x 2 architectures = 60 whole images, and it's virtually guaranteed that 2-3 of the build workflows will fail.

It would be nice to be able to use a local registry mirror when rockcraft build/pack-ing.

aznashwan commented 1 month ago

Related in cause to https://github.com/canonical/rockcraft/issues/654 but functionally independent.