Skaffold needs to propagate `minikube docker-env` to builders

[briandealwis]

@ddobrin was attempting to use Skaffold and Jib with a Spring Boot project. This project includes tests that use testcontainers.org to spin up emulators using docker containers from Java code. The tests fail when run from Skaffold as Skaffold isn't propagating the minikube docker-env to the Jib build, and so the tests try spinning up containers with Docker Desktop's daemon rather than minikube's daemon.

2022-03-03 08:24:58.940  INFO 57968 --- [           main] com.example.QuotesRepositoryTest         : Starting QuotesRepositoryTest using Java 11.0.13 on ddobrin-macbookpro.roam.corp.google.com with PID 57968 (started by ddobrin in /Users/ddobrin/work/dan/modern-java-google-cloud/demo-app)
2022-03-03 08:24:58.941  INFO 57968 --- [           main] com.example.QuotesRepositoryTest         : The following 1 profile is active: "test"
2022-03-03 08:24:59.833  INFO 57968 --- [           main] .s.d.r.c.RepositoryConfigurationDelegate : Bootstrapping Spring Data JPA repositories in DEFAULT mode.
2022-03-03 08:24:59.979  INFO 57968 --- [           main] .s.d.r.c.RepositoryConfigurationDelegate : Finished Spring Data repository scanning in 139 ms. Found 1 JPA repository interfaces.
2022-03-03 08:25:00.551  INFO 57968 --- [           main] com.zaxxer.hikari.HikariDataSource       : HikariPool-1 - Starting...
2022-03-03 08:25:00.669  INFO 57968 --- [           main] o.t.d.DockerClientProviderStrategy       : Loaded org.testcontainers.dockerclient.UnixSocketClientProviderStrategy from ~/.testcontainers.properties, will try it first
2022-03-03 08:25:01.559  INFO 57968 --- [           main] o.t.d.DockerClientProviderStrategy       : Found Docker environment with Environment variables, system properties and defaults. Resolved dockerHost=tcp://127.0.0.1:65029
2022-03-03 08:25:01.559  INFO 57968 --- [           main] org.testcontainers.DockerClientFactory   : Docker host IP address is 127.0.0.1
2022-03-03 08:25:01.615  INFO 57968 --- [           main] org.testcontainers.DockerClientFactory   : Connected to docker: 
  Server Version: 20.10.8
  API Version: 1.41
  Operating System: Ubuntu 20.04.2 LTS
  Total Memory: 11979 MB
2022-03-03 08:25:01.618  INFO 57968 --- [           main] o.t.utility.ImageNameSubstitutor         : Image name substitution will be performed by: DefaultImageNameSubstitutor (composite of 'ConfigurationFileImageNameSubstitutor' and 'PrefixingImageNameSubstitutor')
2022-03-03 08:25:02.354  WARN 57968 --- [containers-ryuk] o.testcontainers.utility.ResourceReaper  : Can not connect to Ryuk at 127.0.0.1:49154

java.net.ConnectException: Connection refused (Connection refused)
    at java.base/java.net.PlainSocketImpl.socketConnect(Native Method) ~[na:na]
    at java.base/java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:399) ~[na:na]
    at java.base/java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:242) ~[na:na]
    at java.base/java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:224) ~[na:na]
    at java.base/java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) ~[na:na]
    at java.base/java.net.Socket.connect(Socket.java:609) ~[na:na]
    at org.testcontainers.utility.ResourceReaper.lambda$null$3(ResourceReaper.java:194) ~[testcontainers-1.16.3.jar:na]
    at org.rnorth.ducttape.ratelimits.RateLimiter.doWhenReady(RateLimiter.java:27) ~[duct-tape-1.0.8.jar:na]
    at org.testcontainers.utility.ResourceReaper.lambda$start$4(ResourceReaper.java:190) ~[testcontainers-1.16.3.jar:na]
    at java.base/java.lang.Thread.run(Thread.java:829) ~[na:na]

We should propagate the minikube docker-env for all tools being invoked from Skaffold.

There are a few workarounds, but none are ideal as the Cloud Run emulation case uses a separate minikube profile and which may not exist until Cloud Code launches the app with the Cloud Run emulator.

1. Configure surefire's environment to with the values of minikube docker-env -p ...
2. Set the environment before launching the IDE: eval $(minikube docker-env -p ...); code .

cc: @sujit-kamireddy

[MSPigl]

@briandealwis I'm running into this issue, but as part of my build process as opposed to unit tests. What are some of those workarounds you mentioned?

[briandealwis]

@MSPigl you should be able to run something like:

(eval $(minikube docker-env); skaffold build ...)

[tejal29]

/cc @OladapoAjala Is this something you are interested in working next? This issue is also in minikube land!

[OladapoAjala]

Yes @tejal29, should be fun. Thanks.

[tejal29]

@OladapoAjala let us know if you are still planning to work on this.

[OladapoAjala]

@tejal29 yes, I'm however engaged a bit right now. Is this high prior?

[dtzoor-s1]

Actually the workaround is to unset the minikube variables before running said command:

(eval $(minikube docker-env --unset); <your command here>)

Is there any update on a better fix for this?