Currently even on master, the integration tests are failing on the latest commit (all I did was trigger a re-run on that commit).
The grader nodes are failing with this error:
grader_1_6991502c1bd8 | 2020-12-17 17:23:38,519 CRITICAL ws._exec_job: grading job failed with exception:
grader_1_6991502c1bd8 | HTTPConnectionPool(host='docker', port=1471): Max retries exceeded with url: /v1.35/images/create?tag=3.5&fromImage=alpine (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f1830e69ba8>: Failed to establish a new connection: [Errno 111] Connection refused',))
Looking deeper, I found this message when the docker daemon is starting. It seems like this was added in a recent/new release (these messages weren't here before) and are causing the docker daemon to not startup properly ("Startup is intentionally being slowed down to show this message").
docker_1_a8c527f3b596 | time="2020-12-17T17:23:36.541469239Z" level=info msg="Starting up"
docker_1_a8c527f3b596 | time="2020-12-17T17:23:36.542003263Z" level=warning msg="Binding to IP address without --tlsverify is insecure and gives root access on this machine to everyone who has access to your network." host="tcp://docker:1471"
docker_1_a8c527f3b596 | time="2020-12-17T17:23:36.542112284Z" level=warning msg="Binding to an IP address, even on localhost, can also give access to scripts run in a browser. Be safe out there!" host="tcp://docker:1471"
docker_1_a8c527f3b596 | time="2020-12-17T17:23:37.543216962Z" level=warning msg="Binding to an IP address without --tlsverify is deprecated. Startup is intentionally being slowed down to show this message" host="tcp://docker:1471"
docker_1_a8c527f3b596 | time="2020-12-17T17:23:37.543257261Z" level=warning msg="Please consider generating tls certificates with client validation to prevent exposing unauthenticated root access to your network" host="tcp://docker:1471"
docker_1_a8c527f3b596 | time="2020-12-17T17:23:37.543272835Z" level=warning msg="You can override this by explicitly specifying '--tls=false' or '--tlsverify=false'" host="tcp://docker:1471"
docker_1_a8c527f3b596 | time="2020-12-17T17:23:37.543285637Z" level=warning msg="Support for listening on TCP without authentication or explicit intent to run without authentication will be removed in the next release" host="tcp://docker:1471"
Based on the recommendations in the logs, I added a --tls=false argument when starting the docker daemon in the integration tests, and the integration tests on this branch now pass. It seems like we need to explicitly specify to not use TLS to avoid the startup slowdown (previously I don't think we had TLS enabled here anyways).
Xiangmingchen
commented
3 years ago
Currently even on master, the integration tests are failing on the latest commit (all I did was trigger a re-run on that commit).
The grader nodes are failing with this error:
Looking deeper, I found this message when the docker daemon is starting. It seems like this was added in a recent/new release (these messages weren't here before) and are causing the docker daemon to not startup properly ("Startup is intentionally being slowed down to show this message").
Based on the recommendations in the logs, I added a
--tls=falseargument when starting the docker daemon in the integration tests, and the integration tests on this branch now pass. It seems like we need to explicitly specify to not use TLS to avoid the startup slowdown (previously I don't think we had TLS enabled here anyways).