search

jupyterhub / binderhub

Run your code in the cloud, with technology so advanced, it feels like magic!
https://binderhub.readthedocs.io
BSD 3-Clause "New" or "Revised" License
2.54k stars 388 forks source link

Remove incorrect docker socket if found #1828

Closed manics closed 7 months ago

manics commented 7 months ago

BinderHub has a well known problem where the docker.socket is created as a directory, which then causes dind to crash: https://github.com/jupyterhub/mybinder.org-deploy/blob/61c0cbedddee92dbcd252e02d9bd02b5a5b94d58/docs/source/operation_guide/common_problems.md#the-docker-in-docker-socket

Given that this problem has existed for years, occurs on multiple deployments, and can result in a prolonged outage until a manual fix is made (most recently today- both Curvenote nodes were hit by this bug simultaneously blocking all builds), I think we should automate the fix.

I originally made this controllable with parameter {dind,pink}.cleanupIncorrectSocketDir but I've removed it in commit d6c66045a0168c8e00f685e1fbf8f21498171392 to reduce the added logic, I'm happy to add it back though.

manics commented 7 months ago

Merging, the failing test is unrelated and the docker socket directory problem has appeared again on mybinder