FR: incremental loader

[josalvatorre]

I witness the following error while trying to run a container test against a tarball created by rules_oci.

➜  mre-container-tar-test git:(main) ✗ bazel test //my_package:empty_file_image_tarball_test
ERROR: /Users/josalvatorre/code/mre-container-tar-test/my_package/BUILD.bazel:26:25: in container_structure_test rule //my_package:empty_file_image_tarball_test:
Traceback (most recent call last):
    File "/private/var/tmp/_bazel_josalvatorre/bb1e2552732b81b1b69870fcd931a2e6/external/container_structure_test~/bazel/container_structure_test.bzl", line 55, column 17, in _structure_test_impl
        fail("when the 'driver' attribute is not 'docker', then the image must be a .tar file")
Error in fail: when the 'driver' attribute is not 'docker', then the image must be a .tar file
ERROR: /Users/josalvatorre/code/mre-container-tar-test/my_package/BUILD.bazel:26:25: Analysis of target '//my_package:empty_file_image_tarball_test' failed
ERROR: Analysis of target '//my_package:empty_file_image_tarball_test' failed; build aborted
INFO: Elapsed time: 0.079s, Critical Path: 0.00s
INFO: 1 process: 1 internal.
ERROR: Build did NOT complete successfully
ERROR: No test targets were found, yet testing was requested

This only happens when testing against a tarball. I can test against the image layer just fine.

I created a simple minimal reproducible example here.

This might just be an issue with container_structure_test. In that case, we can just mention in the documentation that we only support testing against the image layer instead of the tarball.

Sorry if I'm just doing something wrong 😅

[thesayyn]

With 2.0, the default output of oci_tarball is not a tarball anymore. you need a target that depends on oci_tarball to extract the tarball,

FYI, tarballs are expensive to produce and have an immense load on the remote cache and should not be used in this way.

oci_tarball(name = "tarball", ...)

filegroup(
   name = "tarball_tar", 
   srcs = [":tarball"], 
   output_group = "tarball"
)

container_structure_test(
   name = "test",
   image = ":tarball_tar"
)

[alexeagle]

Feels like we should update container_structure_test so it could produce the tar during action execution and pass to the tar driver. Or teach it a new driver against an OCI folder structure.

[thesayyn]

Or teach it a new driver against an OCI folder structure.

this is probably the best idea, an oci driver that only requires a daemon only if there are commandTests but this was a big task last time checked, that's why i shy'ed away/

[josalvatorre]

Thanks for the explanation 🙏 The new driver idea sounds fine to me. For now, I will change my project to simply test against the image. My use case is, thankfully, a low-stakes personal project.

If the team thinks a fix for this isn't worthwhile, then I'd be fine with just adding a caveat to the documentation that clarifies this.

[alexeagle]

I think container-structure-test is just not that useful.

• If you really just want to assert on "structure" like "the image has the right metadata" that's low-value.
• Most users want to assert on something that happens while the container is running, which requires a real SDK for their language like testcontainers.

Thinking in terms of testing as "arrange, act, assert", maybe something like "rules_testcontainers" that gives you the missing "arrange" layer of declarative "this is my test fixture" (in a format that's easy to adopt from docker-compose) and a nice Bazel-y way of tying that to your "act and assert" logic.

[alexeagle]

@gregmagolan convinced me that if we did want to do that, it should just be called oci_test and we just happen to pick testcontainers as a library it builds on.

[alexeagle]

To echo a comment on this closed issue: https://github.com/bazel-contrib/rules_oci/issues/520#issuecomment-1995896969

What we really want is to avoid the non-incremental docker load some.tar behavior since the tar file may be a massive input to the test action. Most testcontainers users just point to a registry. We may need to write a Bazel-specific OCI registry that can run locally as part of an oci_test fixture. It should just rely on blobs already stored in the Bazel cache. Perhaps it should link in or depend on bb-clientd or the new Output File Service in Bazel 7.2 to fetch those blobs by content-address as needed.