search

scrapinghub / shub

Scrapinghub Command Line Client
https://shub.readthedocs.io/
BSD 3-Clause "New" or "Revised" License
125 stars 80 forks source link

Detected error connecting to Docker daemon's host. (minimum version of 1.24 needed) #444

Closed bemayr closed 7 months ago

bemayr commented 7 months ago

Problem Description

After updating Docker I stumbled upon the following error message when trying to deploy a custom image using shub deploy.

Error: 
Detected error connecting to Docker daemon's host.

Please ensure that you have Docker installed, configured and running locally,
that's essential for running shub image command. To check that run command

    docker version

and check its output: it should contain Docker client and server versions and
should not contain any errors. The minimum API Version is: 1.21

You can learn about Docker at https://www.docker.com/.

I immediately stumbled upon https://stackoverflow.com/questions/77893408/persistent-error-deploying-docker-image-to-scrapy-cloud, which explains the exact same problem.

Inspection

So I started inspecting and stumbled upon the line: https://github.com/scrapinghub/shub/blob/5bc420ea2c41c4676ed045b765e05c2027dae48f/shub/image/utils.py#L118-L122 Raising the ShubException in line 122 hides the original error message which is:

docker.errors.APIError: 400 Client Error for http+docker://localhost/v1.21/version: Bad Request ("client version 1.21 is too old. Minimum supported API version is 1.24, please upgrade your client to a newer version")

The documentation of shub states: https://github.com/scrapinghub/shub/blob/5bc420ea2c41c4676ed045b765e05c2027dae48f/docs/configuration.rst?plain=1#L134-L136 but in reality the version is set to: https://github.com/scrapinghub/shub/blob/5bc420ea2c41c4676ed045b765e05c2027dae48f/shub/image/utils.py#L25 which is set to (source):

DEFAULT_DOCKER_API_VERSION = '1.43'
MINIMUM_DOCKER_API_VERSION = '1.21'

Resolution

It seems that the newest Docker version (please see my version info below) does not support v1.21 anymore. I couldn't find any information regarding that problem in the Docker docs. _Manually setting DOCKER_API_VERSION=1.24 resolves the issue._

> docker version
Client:
 Cloud integration: v1.0.35+desktop.10
 Version:           25.0.2
 API version:       1.44
 Go version:        go1.21.6
 Git commit:        29cf629
 Built:             Thu Feb  1 00:18:45 2024
 OS/Arch:           darwin/arm64
 Context:           desktop-linux

Server: Docker Desktop 4.27.1 (136059)
 Engine:
  Version:          25.0.2
  API version:      1.44 (minimum version 1.24)
  Go version:       go1.21.6
  Git commit:       fce6e0c
  Built:            Thu Feb  1 00:23:21 2024
  OS/Arch:          linux/arm64
  Experimental:     false
 containerd:
  Version:          1.6.28
  GitCommit:        ae07eda36dd25f8a1b98dfbf587313b99c0190bb
 runc:
  Version:          1.1.12
  GitCommit:        v1.1.12-0-g51d5e94
 docker-init:
  Version:          0.19.0
  GitCommit:        de40ad0