Docker Compose support doesn't see local Docker images

[ldcasillas-progreso]

I have a project where I've attempted to do the following:

1. Build an image of the project's application, locally (without pushing into a registry);
2. Set up an integration test suite that runs after the image has been built;
3. Have a test case in that suite that uses TestContainer's Compose support to launch a composition that uses the image from step 1.

This however doesn't work, because TestContainers runs Compose inside a container, and it evidently doesn't see the image that's locally built on step 1.

Another symptom of this same problem is that the Compose container, every time it runs, pulls public images afresh from Docker Hub even though they are locally cached on the Docker host.

[outofcoffee]

@ldcasillas-progreso see the withLocalCompose() option I added to the DockerComposeContainer recently.

https://github.com/testcontainers/testcontainers-java/blob/292d0121f7d2bf4cea8b946a9e01ecc566a12f82/core/src/main/java/org/testcontainers/containers/DockerComposeContainer.java#L337

[rnorth]

Hi Luis I'm slightly puzzled as to this because it shouldn't behave this way...

Basically it should work fine because the Docker Compose container process has /var/run/docker.sock volume mounted in, so it should be talking to exactly the same docker daemon that the Compose container is running inside.

The problem you've described could potentially arise if Testcontainers is using a different docker daemon to the one you're using locally (from the command line, I'm guessing?). Is it possible that you've got something like:

• when you run docker-compose locally it's using environment variable config to talk to Docker Machine
• when you run your tests with Testcontainers it's connecting to either a different Docker Machine instance or Docker for Mac

This is the only explanation I can think of right now, but happy to help diagnose this further to get to the bottom of this!

Richard

[monowai]

docker-machine ls shows just the one machine instance and I don't have Docker for Mac installed. I'm running docker-toolbox 1.12.3 on osx.

I've started getting this same error. For me, i think it had been working previously because I inadvertently had a latest tag in the repo which I've now removed. Still looking into a possible root cause

INFO ${sys:PID} --- [rjava-netty-5-3] ?.8.0] : STDERR: Pulling fdstore (flockdata/fd-store:latest)... INFO ${sys:PID} --- [rjava-netty-5-3] ?.8.0] : STDOUT: Pulling repository docker.io/flockdata/fd-store INFO ${sys:PID} --- [rjava-netty-5-3] ?.8.0] : STDERR: Tag latest not found in repository docker.io/flockdata/fd-store

The following images list in my local repo. No idea why it's trying to look remotely.

flockdata/fd-store 0.98.2 03658906237e flockdata/fd-store latest 03658906237e

I tried the .withLocalCompose(true) suggestion but there was no difference. .withPull(false) gets my tests running with the locally build images.

Running docker-compose up -d everything starts fine.

[rnorth]

Hmm, sorry this has languished for so long. I think, based on @monowai's analysis, the preemptive docker-compose pull action we do is the culprit. I'd guess that the docker-compose CLI is smarter than this, and checks the local images prior to issuing a pull command.

Let's go back and look at why we added the pull - I'm sure there was a good reason, so we'll need to work out a sensible balance.

[stale[bot]]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. If you believe this is a mistake, please reply to this comment to keep it open. If there isn't one already, a PR to fix or at least reproduce the problem in a test case will always help us get back on track to tackle this.

[rnorth]

Yikes, the next action is still on me - sorry.

I wonder, actually, if the new PR #924 might help with this, as it will allow docker-compose to run builds of local images itself. We may still need to remove the pre-emptive pull, though.