Support rules_oci

[midnightconman]

As rules_docker is no longer maintained and has some lingering multi-arch issues... we switched to the new rules_oci for image packaging and distribution. It seems that the default resolver doesn't like the new package format for image tarballs (via rule oci_tarball). We get the following error when trying to run rules_k8s targets:

2023/03/31 12:30:39 Unable to publish images: error reading image: unable to load layers from the given parts: unable to build a v1.Layer from the specified parts: unable to load the hashes for compressed layer at : unable to load layer digest from : open : no such file or directory

Here is an example of our bazel build file:

load("@rules_oci//oci:defs.bzl", "oci_image", "oci_tarball")
load("@rules_pkg//:pkg.bzl", "pkg_tar")

pkg_tar(
    name = "tar",
    deps =  [":file"],
)

oci_tarball(
    name = "image_tar",
    image = ":image",
    repotags = ["local:latest"],
)

oci_image(
    name = "image",
    architecture = select({
        "@platforms//cpu:arm64": "arm64",
        "@platforms//cpu:x86_64": "amd64",
    }),
    base = "@base",
    os = "linux",
    tars = [":tar"],
)

k8s_object(
    name = "k8s",
    cluster = "",
    images = {
        "local:latest": ":image_tar",
    },
    kind = "list",
    template = ":manifests.json",
)

I am going to try and create a custom resolver to parse the new manifest.json created by rules_oci... but it would be nice if the default resolver supported these new rules.

Here is an example of a new manifest.json (which is part of the oci_tarball output):

[
  {
    "Config": "blobs/sha256/43439010ce4367cd30de9c43b64171665704c512f2c0e9d63d727233c0f3abd2",
    "RepoTags": ["local:latest"],
    "Layers": [
      "blobs/sha256/895e193edb5191bf66fb5ccb29f5d3659e05eec5953255180cbdd66520e7c517.tar.gz",
      "blobs/sha256/a3e3778621b5e58a0815912eed90e94465c777e2eddd608a5d7200734d003c0f.tar.gz",
      "blobs/sha256/e7cf2c69b92718146707bbea7ebc3259f7611a82166022fd8e87ccc0734d8ea3.tar.gz",
      "blobs/sha256/df40c119df08dddd7210dbf24a59d06a3c0a2b060cfd0425378d47d1e93b106d.tar.gz",
      "blobs/sha256/3b29ea6a27afdac6d3e208f68782496bb55556f762154a72e1bc5537c33ccc2c.tar.gz",
      "blobs/sha256/3997cd6195209aee35967340943da6183326767d9f62df155f3cfbd785b00d9f.tar.gz",
      "blobs/sha256/7e759f975aace530060342a4cbc84811bdbf168cbb0457b7275c8a94ddd5514a.tar.gz",
      "blobs/sha256/ff133072f235dc4955219ef185bc24e87b935c0e74160fa86645f5822981d9e8.tar.gz",
      "blobs/sha256/f9a56094a361b8d24a46359c112f8436885e5665425766f62b6910386e17caf4.tar.gz",
      "blobs/sha256/f188b9e6fee9485edc66809618f7cb82a56f4d283cb25f7f44d8dc2c4b46021a.tar.gz"
    ]
  }
]

Links:

• https://github.com/bazel-contrib/rules_oci
• https://github.com/bazel-contrib/rules_oci/blob/main/docs/tarball.md
• https://github.com/bazel-contrib/rules_oci/blob/main/docs/image.md

[midnightconman]

After a little more digging, it seems the image digest is available from the outputs of the oci_image rule, in index.json. Here is an example:

{
   "schemaVersion": 2,
   "mediaType": "application/vnd.oci.image.index.v1+json",
   "manifests": [
      {
         "mediaType": "application/vnd.docker.distribution.manifest.v2+json",
         "size": 1895,
         "digest": "sha256:397573c2b94603f08933c692ad919aafd185fec73c45c2d6b20f837753977913"
      }
   ]
}

[alexeagle]

+1 - from a scan of this repo I don't see a good reason that it depends on rules_docker at all. I think it should just be properly layered to know nothing about how you construct your images.

[mrmeku]

@midnightconman FYI, the output of oci_image is always going to be a directory artifact that follows the layout specified by the OCI image-spec https://github.com/opencontainers/image-spec/blob/main/image-layout.md

That should give us a reliable way to parse the index.json file since we know its media type: application/vnd.oci.image.index.v1+json

[dimatosaurus]

Any success here getting the k8s rule to work with oci_image? I switched from the docker rules because of go dependency problems, but now broke my local development pipeline.

[alexeagle]

Sorry, no success because this repo appears to be totally unmaintained, after trying in a few ways I haven't been able to reach anyone. Likely it's going to be archived.

[fparga]

@alexeagle is there any chance we might see an aspect-build/rules_k8s one day?

[alexeagle]

I doubt it, unless we find some funding from the community or from one of our clients.

We've been modelling the CI/CD handoff as Continuous Delivery rather than Continuous Deployment: "Bazel has to create artifacts and deliver them somewhere, a different tool then picks them up and promotes them to the next environment dev/staging/prod" https://docs.aspect.build/v/workflows/delivery and when modeled that way, something that can do a k8s deploy is on the "other side of the fence".

Of course that's not a reason that something like rules_k8s shouldn't exist, it's just less urgent for us to jump in and drive.