testcontainers / testcontainers-java

Testcontainers is a Java library that supports JUnit tests, providing lightweight, throwaway instances of common databases, Selenium web browsers, or anything else that can run in a Docker container.
https://testcontainers.org
MIT License
8.02k stars 1.65k forks source link

Image Name Substitution not applied for DockerComposeContainers #4605

Open psxpaul opened 3 years ago

psxpaul commented 3 years ago

I have a set of tests which create a DockerComposeContainer on startup. I ran into the below error when the tests run on a CI server:

com.github.dockerjava.api.exception.InternalServerErrorException: Status 500: {"message":"toomanyrequests: You have reached your pull rate limit. You may increase the limit by authenticating and upgrading: https://www.docker.com/increase-rate-limit"}

First, I tried setting a hub.image.name.prefix to point to my company's local registry, but continued to run into the error. I next tried creating a custom ImageNameSubstitutor to redirect images to my company's local registry, but that did not seem to help either. I then tried changing all image names in my docker-compose file to point at the local registry, and that seemed to resolve my problems.

I could be wrong, but I suspect this is because the image substitution mechanism is called from RemoteDockerImage, which is not used when parsing docker-compose files. My build logs seem to indicate that the ryuk and socat images are correctly being pulled from the local registry, but not the images referenced in the compose file.

Is this intended behavior? I don't see any references to docker-compose on the documentation page about image substitution.

kiview commented 3 years ago

Hey @psxpaul, thanks for bringing this to our attention, and sorry that you had to dig into the sources to understand what is happening.

Image name substitution is indeed not supported for DockerComposeContainer. This was a conscious decision since it is complex and involves many edge cases. Nevertheless, this should indeed be added to the documentation and we will follow up on that.

greenstevester commented 2 years ago

Hi there,

Also got the same issue when setting TESTCONTAINERS_HUB_IMAGE_NAME_PREFIX env var with my company docker repo: my-company.net/

It removed the trailing backslash and of course could not resolve the host.

I created an ExampleImageNameSubstitutor per https://www.testcontainers.org/features/image_name_substitution/

` public class ExampleImageNameSubstitutor extends ImageNameSubstitutor {

@Override
public DockerImageName apply(DockerImageName original) {
    // If we detected our internal docker as docker repo, append extra / as it is currently missing
    // due to a bug https://github.com/testcontainers/testcontainers-java/issues/4605
    // in testcontainers.org.
    String tmp = original.asCanonicalNameString();
    int index = tmp.indexOf("my-company.net");
    if (index>-1) {
        tmp = "my-company.net/" + tmp.substring(index+14);
    }
    DockerImageName dockerImageName = DockerImageName.parse(tmp);
    return dockerImageName;
}

@Override
protected String getDescription() {
    // used in logs
    return "example image name substitutor";
}

} ` Which is a workaround for the issue. Regards, Steve