Allow running without network

ErikSchierboom commented 3 years ago

If I workaround https://github.com/exercism/kotlin-test-runner/issues/14 by removing the --readonly option in the ./bin/run-in-docker.sh script and then run:

./bin/run-in-docker.sh hello-world /home/erik/exercism-repos/kotlin-test-runner/examples/full /home/erik/exercism-repos/kotlin-test-runner/examples/full

I get another error"

Use 'docker scan' to run Snyk tests against images to find vulnerabilities and learn how to fix them
Parsed arguments: LaunchArguments(exerciseSlug=hello-world, solutionsDir=/solution, resultFile=/output/results.json)
Running gradle
Gradle finished with exit code 1
=== Log START ===
Downloading https://services.gradle.org/distributions/gradle-6.0.1-bin.zip

Exception in thread "main" java.net.UnknownHostException: services.gradle.org
        at java.base/java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:220)
        at java.base/java.net.SocksSocketImpl.connect(SocksSocketImpl.java:403)
        at java.base/java.net.Socket.connect(Socket.java:609)
        at java.base/sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:285)
        at java.base/sun.security.ssl.BaseSSLSocketImpl.connect(BaseSSLSocketImpl.java:173)
        at java.base/sun.net.NetworkClient.doConnect(NetworkClient.java:182)
        at java.base/sun.net.www.http.HttpClient.openServer(HttpClient.java:474)
        at java.base/sun.net.www.http.HttpClient.openServer(HttpClient.java:569)
        at java.base/sun.net.www.protocol.https.HttpsClient.<init>(HttpsClient.java:265)
        at java.base/sun.net.www.protocol.https.HttpsClient.New(HttpsClient.java:372)
        at java.base/sun.net.www.protocol.https.AbstractDelegateHttpsURLConnection.getNewHttpClient(AbstractDelegateHttpsURLConnection.java:191)
        at java.base/sun.net.www.protocol.http.HttpURLConnection.plainConnect0(HttpURLConnection.java:1187)
        at java.base/sun.net.www.protocol.http.HttpURLConnection.plainConnect(HttpURLConnection.java:1081)
        at java.base/sun.net.www.protocol.https.AbstractDelegateHttpsURLConnection.connect(AbstractDelegateHttpsURLConnection.java:177)
        at java.base/sun.net.www.protocol.http.HttpURLConnection.getInputStream0(HttpURLConnection.java:1587)
        at java.base/sun.net.www.protocol.http.HttpURLConnection.getInputStream(HttpURLConnection.java:1515)
        at java.base/sun.net.www.protocol.https.HttpsURLConnectionImpl.getInputStream(HttpsURLConnectionImpl.java:250)
        at org.gradle.wrapper.Download.downloadInternal(Download.java:67)
        at org.gradle.wrapper.Download.download(Download.java:52)
        at org.gradle.wrapper.Install$1.call(Install.java:62)
        at org.gradle.wrapper.Install$1.call(Install.java:48)
        at org.gradle.wrapper.ExclusiveFileAccessManager.access(ExclusiveFileAccessManager.java:69)
        at org.gradle.wrapper.Install.createDist(Install.java:48)
        at org.gradle.wrapper.WrapperExecutor.execute(WrapperExecutor.java:107)
        at org.gradle.wrapper.GradleWrapperMain.main(GradleWrapperMain.java:63)

=== Log END ===

It appears that the test runner is not able to run without networking, which is a hard requirement for test runners. The way tracks should solve this problem is to download all the dependencies in advance in the Dockerfile, which has access to the network. It is only when the Docker image is run that there is no network.

For inspiration, check the https://github.com/exercism/java-test-runner and https://github.com/exercism/groovy-test-runner/ repos to see how they are handling it. https://github.com/exercism/groovy-test-runner/

ErikSchierboom commented 3 years ago

(cc @exercism/kotlin)

dector commented 3 years ago

Originally I was targeting for smaller image so I didn't include all Gradle caches. Will rework Dockerfile.

ErikSchierboom commented 3 years ago

Brilliant, thanks!

geoffreywiseman commented 3 years ago

@dector How goes?

ErikSchierboom commented 3 years ago

Yeah, I have the same question :) @dector is there anything I can help with?

dector commented 3 years ago

@geoffreywiseman @ErikSchierboom

I would appreciate some help here. Lack of proper Docker knowledge bit me in the arse. 😞

The plan is to cache project dependencies into Gradle cache so I'll be able to run in --offline mode. However, I faced some issue with Docker: when I'm resolving dependencies using cache-warmup project - instead of creating new layer, results of this RUN step are thrown away and resulting build container has no cache inside.

Please tell me where I'm wrong and what should I do instead?

Here I'm resolving dependencies to cache: click.

Here I'm expecting them to be copied to result container: click.

What I've tried already:

Running gradle resolveDependencies in running container - it works, caches are stored as expected. At first I thought that I have issues with gradle environment variables, folders etc. And Gradle just keeping cache somewhere in /tmp. Spend lots of weekend hours. But I was wrong - Gradle works fine.
Warming-up cache in result container. I basically removed two-staged build and made it one-staged. But Docker still not creating new layer.
Running docker build without --rm. Didn't work either. As I understand, created container is kept in local FS, but new layer is not created.

Do you have any ideas what to do in this case?

dector commented 3 years ago

Here is dive into container: Screenshot_2021-06-12_14-54-32

As we can see - copied /home/gradle/.gradle that should contain all caches is empty.

However if you'll copy cache-warmup files into container, run it with docker --entrypoint=/bin/bash exec -it and do gradle resolveDependencies; ls /home/gradle/.gradle - you will see cache directory there.

ErikSchierboom commented 3 years ago

@dector I've checked-out your branch locally. If I then change COPY cache-warmup/ ./ to COPY cache-warmup ./cache-warmup and then test using ./bin/run-in-docker.sh hello-world $PWD/examples/full $PWD/examples/full, I can see a results.json file being produced that looks like this:

{
  "status": "fail",
  "message": "",
  "tests": [
    {
      "name": "helloWorldTest()",
      "status": "fail",
      "message": "org.opentest4j.AssertionFailedError: expected: <Hello, World!> but was: <Hello, World!1>",
      "output": ""
    },
    {
      "name": "passing test()",
      "status": "pass",
      "message": "",
      "output": ""
    }
  ]
}

So your branch looks to be working with the above change! The only thing that I noticed was that compilation was quite slow. I'm afraid that the current test runner won't be able to finish within the maximum of 10 seconds. Here are the timing results of the docker run command execute in the above example:

real    0m24.173s
user    0m0.181s
sys     0m0.084s

Is there anything we can do to speed-up compilation?

dector commented 3 years ago

Thanks. I will check your solution tomorrow and compilation time.

I had an idea not to use Gradle and run Kotlin compiler directly. This will not be as flexible, but our exercises are very simple.

ErikSchierboom commented 3 years ago

I had an idea not to use Gradle and run Kotlin compiler directly. This will not be as flexible, but our exercises are very simple.

I've done the exact same thing for the C# test runner. In my case, it was definitely worth the trade-off, as execution time was more than halved.

dector commented 3 years ago

It makes sense to try then.

Do you know something about available RAM on test running machine?

ErikSchierboom commented 3 years ago

The containers run with 3GB of memory.

exercism / kotlin-test-runner

Allow running without network #16