bazel-contrib / rules_oci

Bazel rules for building OCI containers
Apache License 2.0
302 stars 158 forks source link

Downloading an OCI image from public AWS ECR using `tag` causes `mediaType` exception #496

Open Wito-1 opened 9 months ago

Wito-1 commented 9 months ago

It looks like when trying to download a debian image from AWS ECR: public.ecr.aws/debian/debian:latest, however, it looks like the manifest does not contain a mediaType field at the toplevel:

$ bazel query @debian//...
...
ERROR: An error occurred during the fetch of repository 'rules_oci~1.7.0~oci~debian':
   Traceback (most recent call last):
    File "/home/juan/.cache/bazel/_bazel_juan/72d38e88d392a1c0f4981c65e0bb0d1e/external/rules_oci~1.7.0/oci/private/pull.bzl", line 462, column 16, in _oci_alias_impl
        if manifest["mediaType"] in _SUPPORTED_MEDIA_TYPES["index"]:
Error: key "mediaType" not found in dictionary
ERROR: <builtin>: fetching oci_alias rule //:rules_oci~1.7.0~oci~debian: Traceback (most recent call last):
    File "/home/juan/.cache/bazel/_bazel_juan/72d38e88d392a1c0f4981c65e0bb0d1e/external/rules_oci~1.7.0/oci/private/pull.bzl", line 462, column 16, in _oci_alias_impl
        if manifest["mediaType"] in _SUPPORTED_MEDIA_TYPES["index"]:
Error: key "mediaType" not found in dictionary

Here's my MODULE.bazel file for reference:

bazel_dep(name = "rules_oci", version = "1.7.0")

oci = use_extension("@rules_oci//oci:extensions.bzl", "oci")

# Declare external images you need to pull, for example: 
oci.pull(
    name = "debian",
    # 'latest' is not reproducible, but it's convenient.
    # During the build we print a WARNING message that includes recommended 'digest' and 'platforms'
    # values which you can use here in place of 'tag' to pin for reproducibility.
    tag = "latest",
    image = "public.ecr.aws/debian/debian",
    platforms = [
        "linux/amd64",
        "linux/arm/v7",
        "linux/arm64/v8",
    ],
)

# For each oci.pull call, repeat the "name" here to expose them as dependencies.
use_repo(oci, "debian")

Running on: bazel 6.4.0

It seems to be able to pull fine if I provide the digest:

oci.pull(
    name = "debian",
    # 'latest' is not reproducible, but it's convenient.
    # During the build we print a WARNING message that includes recommended 'digest' and 'platforms'
    # values which you can use here in place of 'tag' to pin for reproducibility.
    tag = "latest",
    digest = "sha256:689ff332097aea20e4beff21741089a9a2957c5abf127ee2829b530b83ba3a12",
    image = "debian",
    platforms = [
        "linux/amd64",
        "linux/arm/v7",
        "linux/arm64/v8",
    ],
)
$ bazel query @debian//...
...
buildozer 'set digest "sha256:79becb70a6247d277b59c09ca340bbe0349af6aacb5afa90ec349528b53ce2c9"' 'remove tag' 'remove platforms' 'add platforms "linux/amd64" "linux/arm/v5" "linux/arm/v7" "linux/arm64/v8" "linux/386" "linux/mips64le" "linux/ppc64le" "linux/s390x"' MODULE.bazel:debian

@debian//:debian
mikebauer commented 3 months ago

I'm also experiencing this issue. I'm unable to get the work-around working though. Are you sure it's not just pulling from dockerhub in that case, since the image in the working example is "debian" and not "public.ecr.aws/debian/debian"?

alexeagle commented 2 months ago

Seems like something is broken about the metadata, where we can't distinguish image from imageIndex. Maybe this thing isn't following the specification.

blockjon-dd commented 1 month ago

I'm having a similar problem. Before rules_oci, my image indexes had this sort of json manifest:

{
   "schemaVersion": 2,
   "mediaType": "application/vnd.docker.distribution.manifest.list.v2+json",
   "manifests": [
      {
         "mediaType": "application/vnd.docker.distribution.manifest.v2+json",
         "size": 4492,
         "digest": "sha256:a97bcd99f2c5d7033d3b80c203b90557d4dacdea2d1c93b9465ccfe64c74f33f",
         "platform": {
            "architecture": "amd64",
            "os": "linux"
         }
      },
      {
         "mediaType": "application/vnd.docker.distribution.manifest.v2+json",
         "size": 4492,
         "digest": "sha256:32250033d499ec2afc1a44096fc72bf8cc569a7797b4dd6cb515ae06563e3127",
         "platform": {
            "architecture": "arm64",
            "os": "linux"
         }
      }
   ]
}

under rules_oci they look more like this:

{
  "schemaVersion": 2,
  "manifests": [
    {
      "mediaType": "application/vnd.oci.image.index.v1+json",
      "size": 648,
      "digest": "sha256:77b10ee3e650f78f19bf8b18e23a77832db804a1897b0bbca1badca403786b7d"
    }
  ]
}

I'm getting an error that looks like this in my app logic:

[dd:e17142] Error occurred while tagging demohelloworldpython image xxxx with tag 2.0.0 in registry 23432432423: Invalid parameter at 'ImageManifest' failed to satisfy constraint: 'Invalid JSON syntax' (Service: Ecr, Status Code: 400, Request ID: 0869a62c-8dcf-4207-93e9-f7638daf8c50)