[Enhancement]: run container directly from image tarball

[nmoroze]

Proposal

Hi, thanks for the great package!

I have a use case where it would be helpful to directly run an image from a path to a container tarball, like the type generated by rules_oci. This would facilitate the workflow described in https://blog.aspect.dev/integration-testing-oci#heading-example without having to perform the extra step of loading the tarball into Podman or Docker's local image storage.

I was thinking one way to support this from a UX perspective would be to support the "transport" protocol described here: https://docs.podman.io/en/latest/markdown/podman-run.1.html#image, so you could specify docker-archive:/path/to/tar or oci-archive:/path/to/tar as the Image field in a ContainerRequest.

I'm curious if you think this would be a reasonable feature to add?

It would also be useful to support supplying a path to the reaper container this way. I'm happy to open a separate issue for that if you think it's viable/worth discussing.

[mdelapenya]

Hi @nmoroze is this capability of the FromDockerfile struct what you need?

ContextArchive io.Reader    // the tar archive file to send to docker that contains the build context

You can find more info about that here: https://golang.testcontainers.org/features/build_from_dockerfile/#dynamic-build-context

[nmoroze]

Thanks for the pointer! That's not quite what I'm looking for though. If I understand correctly, it seems like that feature lets you provide a tarball that contains the set of files to be added to a container (with the container otherwise being defined by a Dockerfile), whereas I'm looking to load a tarball in the form defined by the OCI image layout spec and/or the format produced by docker save (I think either would be fine).

[kiview]

Hey @nmoroze, could you please elaborate your use case, why you need to do it like this? Do you basically want to be able to run docker load from tc-go? Because I don't see how we can avoid doing loading the images, if Docker itself does not provide the capability to run a container from an image tarball without doing a load.

[nmoroze]

@kiview Our use case is running containerized integration tests using Bazel, which wants tests to be as hermetic as possible, i.e. eliminating dependence on the state of the host machine. Our build process generates container images for our services as tarballs, and it'd be nice to be able to load/run these in a way where we:

a) know we're running the correct thing. However unlikely, loading an image then referencing it by label could have a race if a user loaded a different image with the same tag in between the load/run step.

b) avoid polluting the user's local image storage

Do you basically want to be able to run docker load from tc-go?

If nothing else, I think this would be nice. Otherwise, we have to either shell out to the container client directly, or bring in another package that provides bindings to the container client, which feels slightly less clean than having testcontainers handle all the container client interaction. If testcontainers handles the load it'd also be nice if it could manage unloading/have the reaper clear out these images from local storage, which takes care of point (b) above.

I don't see how we can avoid doing loading the images, if Docker itself does not provide the capability to run a container from an image tarball without doing a load.

I believe Podman supports this, which is the container client we're using: https://docs.podman.io/en/latest/markdown/podman-run.1.html#image. I think running directly from tar using this functionality would be ideal, but if Docker can't do this, I'd understand if you want to minimize divergence of Docker vs Podman codepaths.

I did also note that page says For remote Podman, including Mac and Windows (excluding WSL2) machines, docker is the only allowed transport. I'm not sure if that means this idea is a non-starter given how testcontainers interacts with Podman anyways.

[nmoroze]

Another thought - is there a way to tag an image that is loaded outside of testcontainers so that the Reaper container would delete it after the test runs? I think this would help accomplish goal (b) above.

[kochkarev]

Hi @nmoroze! Didn't you find out how to resolve this issue? We have same problem - our applications are built with Bazel and we want to launch them with testcontainers inside integration tests. The only way out I see here - parse ImageLoadResponse.Body to find out image sha, but it's rather weird..

[nickbreen]

Same boat, with Bazel, except with tc-java.

Here's an example of DIY'ing the docker load (with python). https://blog.aspect.build/integration-testing-oci

An equivalent to DockerImageName.parse(String) like OCIImageArchive.load(Path) is kind of what I expect.

Sorry, just realised this was tc-go not tc-java.