Closed wiesbadener closed 1 year ago
@wiesbadener thank you for reporting this. This might be related to Docker not being accessible by non-root users on your system, but it's difficult to tell.
The piece of ZenML code that is reporting that error is quite simple and you should be able to run it yourself to get more information into the error:
python -c "from docker.client import DockerClient; docker_client = DockerClient.from_env(); docker_client.ping()"
The stack trace should give more information into why connecting to Docker with the python client fails.
@wiesbadener Did that workaround help at all?
Thanks @stefannica for your reply. Sorry for the late response.
I ran your code and get the following errors. Still investigating the reasons.
$ python -c "from docker.client import DockerClient; docker_client = DockerClient.from_env(); docker_client.ping()"
Traceback (most recent call last):
File "~/miniconda3/envs/mlenv/lib/python3.9/site-packages/urllib3/connectionpool.py", line 703, in urlopen
httplib_response = self._make_request(
File "~/miniconda3/envs/mlenv/lib/python3.9/site-packages/urllib3/connectionpool.py", line 398, in _make_request
conn.request(method, url, **httplib_request_kw)
File "~/miniconda3/envs/mlenv/lib/python3.9/http/client.py", line 1285, in request
self._send_request(method, url, body, headers, encode_chunked)
File "~/miniconda3/envs/mlenv/lib/python3.9/http/client.py", line 1331, in _send_request
self.endheaders(body, encode_chunked=encode_chunked)
File "~/miniconda3/envs/mlenv/lib/python3.9/http/client.py", line 1280, in endheaders
self._send_output(message_body, encode_chunked=encode_chunked)
File "~/miniconda3/envs/mlenv/lib/python3.9/http/client.py", line 1040, in _send_output
self.send(msg)
File "~/miniconda3/envs/mlenv/lib/python3.9/http/client.py", line 980, in send
self.connect()
File "~/miniconda3/envs/mlenv/lib/python3.9/site-packages/docker/transport/unixconn.py", line 30, in connect
sock.connect(self.unix_socket)
FileNotFoundError: [Errno 2] No such file or directory
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "~/miniconda3/envs/mlenv/lib/python3.9/site-packages/requests/adapters.py", line 489, in send
resp = conn.urlopen(
File "~/miniconda3/envs/mlenv/lib/python3.9/site-packages/urllib3/connectionpool.py", line 787, in urlopen
retries = retries.increment(
File "~/miniconda3/envs/mlenv/lib/python3.9/site-packages/urllib3/util/retry.py", line 550, in increment
raise six.reraise(type(error), error, _stacktrace)
File "~/miniconda3/envs/mlenv/lib/python3.9/site-packages/urllib3/packages/six.py", line 769, in reraise
raise value.with_traceback(tb)
File "~/miniconda3/envs/mlenv/lib/python3.9/site-packages/urllib3/connectionpool.py", line 703, in urlopen
httplib_response = self._make_request(
File "~/miniconda3/envs/mlenv/lib/python3.9/site-packages/urllib3/connectionpool.py", line 398, in _make_request
conn.request(method, url, **httplib_request_kw)
File "~/miniconda3/envs/mlenv/lib/python3.9/http/client.py", line 1285, in request
self._send_request(method, url, body, headers, encode_chunked)
File "~/miniconda3/envs/mlenv/lib/python3.9/http/client.py", line 1331, in _send_request
self.endheaders(body, encode_chunked=encode_chunked)
File "~/miniconda3/envs/mlenv/lib/python3.9/http/client.py", line 1280, in endheaders
self._send_output(message_body, encode_chunked=encode_chunked)
File "~/miniconda3/envs/mlenv/lib/python3.9/http/client.py", line 1040, in _send_output
self.send(msg)
File "~/miniconda3/envs/mlenv/lib/python3.9/http/client.py", line 980, in send
self.connect()
File "~/miniconda3/envs/mlenv/lib/python3.9/site-packages/docker/transport/unixconn.py", line 30, in connect
sock.connect(self.unix_socket)
urllib3.exceptions.ProtocolError: ('Connection aborted.', FileNotFoundError(2, 'No such file or directory'))
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "~/miniconda3/envs/mlenv/lib/python3.9/site-packages/docker/api/client.py", line 214, in _retrieve_server_version
return self.version(api_version=False)["ApiVersion"]
File "~/miniconda3/envs/mlenv/lib/python3.9/site-packages/docker/api/daemon.py", line 181, in version
return self._result(self._get(url), json=True)
File "~/miniconda3/envs/mlenv/lib/python3.9/site-packages/docker/utils/decorators.py", line 46, in inner
return f(self, *args, **kwargs)
File "~/miniconda3/envs/mlenv/lib/python3.9/site-packages/docker/api/client.py", line 237, in _get
return self.get(url, **self._set_request_timeout(kwargs))
File "~/miniconda3/envs/mlenv/lib/python3.9/site-packages/requests/sessions.py", line 600, in get
return self.request("GET", url, **kwargs)
File "~/miniconda3/envs/mlenv/lib/python3.9/site-packages/requests/sessions.py", line 587, in request
resp = self.send(prep, **send_kwargs)
File "~/miniconda3/envs/mlenv/lib/python3.9/site-packages/requests/sessions.py", line 701, in send
r = adapter.send(request, **kwargs)
File "~/miniconda3/envs/mlenv/lib/python3.9/site-packages/requests/adapters.py", line 547, in send
raise ConnectionError(err, request=request)
requests.exceptions.ConnectionError: ('Connection aborted.', FileNotFoundError(2, 'No such file or directory'))
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "<string>", line 1, in <module>
File "~/miniconda3/envs/mlenv/lib/python3.9/site-packages/docker/client.py", line 96, in from_env
return cls(
File "~/miniconda3/envs/mlenv/lib/python3.9/site-packages/docker/client.py", line 45, in __init__
self.api = APIClient(*args, **kwargs)
File "~/miniconda3/envs/mlenv/lib/python3.9/site-packages/docker/api/client.py", line 197, in __init__
self._version = self._retrieve_server_version()
File "~/miniconda3/envs/mlenv/lib/python3.9/site-packages/docker/api/client.py", line 221, in _retrieve_server_version
raise DockerException(
docker.errors.DockerException: Error while fetching server API version: ('Connection aborted.', FileNotFoundError(2, 'No such file or directory'))
@wiesbadener I hope you figured out how to manage Docker on your system as a non-root user. If not, please see the following: https://docs.docker.com/engine/install/linux-postinstall/#manage-docker-as-a-non-root-user.
In any case, seeing as this is not a ZenML issue but rather a Docker installation issue on your machine, I would recommend we close this issue.
Yes, it’s a docker related issue. Thanks! Closing it.
@wiesbadener thank you !
Contact Details [Optional]
System Information
Unable to find ZenML repository in your current working directory (/home/user/foo/bar) or any parent directories. If you want to use an existing repository which is in a different location, set the environment variable 'ZENML_REPOSITORY_PATH'. If you want to create a new repository, run zenml init. Running without an active repository root. ZENML_LOCAL_VERSION: 0.38.0 ZENML_SERVER_VERSION: 0.38.0 ZENML_SERVER_DATABASE: sqlite ZENML_SERVER_DEPLOYMENT_TYPE: local ZENML_CONFIG_DIR: /home/user/.config/zenml ZENML_LOCAL_STORE_DIR: /home/user/.config/zenml/local_stores ZENML_SERVER_URL: http://127.0.0.1:8237 ZENML_ACTIVE_REPOSITORY_ROOT: None PYTHON_VERSION: 3.9.16 ENVIRONMENT: native SYSTEM_INFO: {'os': 'linux', 'linux_distro': 'ubuntu', 'linux_distro_like': 'debian', 'linux_distro_version': '22.04'} ACTIVE_WORKSPACE: default ACTIVE_STACK: default ACTIVE_USER: default TELEMETRY_STATUS: enabled ANALYTICS_CLIENT_ID: e65e267b-452f-4bc0-a29a-53b9b76a26d9 ANALYTICS_USER_ID: 193c1ec4-ff4d-4dbc-9de5-11be8522f668 ANALYTICS_SERVER_ID: e65e267b-452f-4bc0-a29a-53b9b76a26d9 INTEGRATIONS: ['huggingface', 'kaniko', 'pillow', 'pytorch', 'scipy', 'sklearn'] PACKAGES: {'authlib': '1.2.0', 'deprecated': '1.2.13', 'events': '0.4', 'flask': '2.3.1', 'gitpython': '3.1.31', 'jinja2': '3.1.2', 'mako': '1.2.4', 'markdown': '3.3.7', 'markupsafe': '2.1.2', 'pillow': '9.5.0', 'pyjwt': '2.6.0', 'pymupdf': '1.22.2', 'pymysql': '1.0.3', 'pysocks': '1.7.1', 'pyyaml': '5.4.1', 'pygments': '2.15.1', 'sparqlwrapper': '2.0.0', 'sqlalchemy': '1.4.41', 'sqlalchemy-utils': '0.38.3', 'send2trash': '1.8.2', 'werkzeug': '2.3.0', 'aiohttp': '3.8.4', 'aiohttp-cors': '0.7.0', 'aiorwlock': '1.3.0', 'aiosignal': '1.3.1', 'alembic': '1.8.1', 'analytics-python': '1.4.post1', 'anyio': '3.6.2', 'appdirs': '1.4.4', 'argon2-cffi': '21.3.0', 'argon2-cffi-bindings': '21.2.0', 'arrow': '1.2.3', 'asgiref': '3.6.0', 'astroid': '2.15.4', 'asttokens': '2.2.1', 'async-generator': '1.10', 'async-timeout': '4.0.2', 'attrs': '23.1.0', 'azure-ai-formrecognizer': '3.3.0b1', 'azure-common': '1.1.28', 'azure-core': '1.26.4', 'backcall': '0.2.0', 'backoff': '1.10.0', 'backports.functools-lru-cache': '1.6.4', 'bcrypt': '4.0.1', 'beautifulsoup4': '4.12.2', 'beir': '1.0.1', 'black': '23.3.0', 'bleach': '6.0.0', 'blessed': '1.20.0', 'blinker': '1.6.2', 'boilerpy3': '1.0.6', 'cachetools': '5.3.0', 'canals': '0.0.3', 'cattrs': '22.2.0', 'certifi': '2022.12.7', 'cffi': '1.15.1', 'cfgv': '3.3.1', 'chardet': '5.1.0', 'charset-normalizer': '3.1.0', 'click': '8.1.3', 'click-params': '0.3.0', 'cloudpickle': '2.2.1', 'cmake': '3.26.3', 'colorama': '0.4.6', 'coloredlogs': '15.0.1', 'colorful': '0.5.5', 'commonmark': '0.9.1', 'contourpy': '1.0.7', 'coverage': '7.2.3', 'cryptography': '40.0.2', 'cycler': '0.11.0', 'databind': '1.5.3', 'databind.core': '1.5.3', 'databind.json': '1.5.3', 'databricks-cli': '0.17.6', 'datasets': '2.11.0', 'debugpy': '1.5.1', 'decorator': '5.1.1', 'defusedxml': '0.7.1', 'dill': '0.3.6', 'distlib': '0.3.6', 'distro': '1.8.0', 'dnspython': '2.3.0', 'docker': '6.0.1', 'docopt': '0.6.2', 'docspec': '2.1.2', 'docspec-python': '2.1.2', 'docstring-parser': '0.11', 'ecdsa': '0.18.0', 'elasticsearch': '7.9.1', 'entrypoints': '0.4', 'exceptiongroup': '1.1.1', 'executing': '1.2.0', 'faiss-cpu': '1.7.2', 'farm-haystack': '1.15.1', 'fastapi': '0.75.2', 'fastapi-utils': '0.2.1', 'fastjsonschema': '2.16.3', 'ffmpeg-python': '0.2.0', 'filelock': '3.12.0', 'flatbuffers': '23.3.3', 'fonttools': '4.39.3', 'fqdn': '1.5.1', 'frozenlist': '1.3.3', 'fsspec': '2023.4.0', 'future': '0.18.3', 'ghp-import': '2.1.0', 'gitdb': '4.0.10', 'google-api-core': '2.11.0', 'google-auth': '2.17.3', 'googleapis-common-protos': '1.59.0', 'gpustat': '1.1', 'greenlet': '2.0.2', 'grpcio': '1.37.1', 'grpcio-tools': '1.37.1', 'gunicorn': '20.1.0', 'h11': '0.14.0', 'httplib2': '0.19.1', 'httptools': '0.5.0', 'huggingface-hub': '0.14.1', 'humanfriendly': '10.0', 'identify': '2.5.23', 'idna': '3.4', 'importlib-metadata': '6.6.0', 'importlib-resources': '5.12.0', 'inflect': '6.0.4', 'iniconfig': '2.0.0', 'ipykernel': '6.15.0', 'ipython': '8.12.0', 'ipython-genutils': '0.2.0', 'ipywidgets': '7.7.5', 'isodate': '0.6.1', 'isoduration': '20.11.0', 'isort': '5.12.0', 'itsdangerous': '2.1.2', 'jarowinkler': '1.2.3', 'jedi': '0.18.2', 'joblib': '1.2.0', 'jsonpointer': '2.3', 'jsonschema': '4.17.3', 'jupyter-client': '8.2.0', 'jupyter-core': '5.3.0', 'jupyter-events': '0.6.3', 'jupyter-server': '2.5.0', 'jupyter-server-terminals': '0.4.4', 'jupytercontrib': '0.0.7', 'jupyterlab-pygments': '0.2.2', 'jupyterlab-widgets': '1.1.4', 'kiwisolver': '1.4.4', 'langdetect': '1.0.9', 'lazy-object-proxy': '1.9.0', 'lit': '16.0.2', 'llvmlite': '0.39.1', 'loguru': '0.7.0', 'lxml': '4.9.2', 'matplotlib': '3.7.1', 'matplotlib-inline': '0.1.6', 'mccabe': '0.7.0', 'mergedeep': '1.3.4', 'mistune': '2.0.5', 'mkdocs': '1.4.2', 'mlflow': '2.3.0', 'mmh3': '3.0.0', 'monotonic': '1.6', 'more-itertools': '9.1.0', 'mpmath': '1.3.0', 'msgpack': '1.0.5', 'msrest': '0.7.1', 'multidict': '6.0.4', 'multiprocess': '0.70.14', 'mypy': '1.2.0', 'mypy-extensions': '1.0.0', 'nbclassic': '1.0.0', 'nbclient': '0.7.4', 'nbconvert': '7.3.1', 'nbformat': '5.8.0', 'nest-asyncio': '1.5.6', 'networkx': '3.1', 'nltk': '3.8.1', 'nodeenv': '1.7.0', 'notebook': '6.5.4', 'notebook-shim': '0.2.3', 'nr.util': '0.8.12', 'num2words': '0.5.12', 'numba': '0.56.4', 'numpy': '1.23.5', 'nvidia-cublas-cu11': '11.10.3.66', 'nvidia-cuda-cupti-cu11': '11.7.101', 'nvidia-cuda-nvrtc-cu11': '11.7.99', 'nvidia-cuda-runtime-cu11': '11.7.99', 'nvidia-cudnn-cu11': '8.5.0.96', 'nvidia-cufft-cu11': '10.9.0.58', 'nvidia-curand-cu11': '10.2.10.91', 'nvidia-cusolver-cu11': '11.4.0.1', 'nvidia-cusparse-cu11': '11.7.4.91', 'nvidia-ml-py': '11.525.112', 'nvidia-nccl-cu11': '2.14.3', 'nvidia-nvtx-cu11': '11.7.91', 'oauthlib': '3.2.2', 'onnx': '1.13.1', 'onnxruntime': '1.14.1', 'onnxruntime-tools': '1.7.0', 'openai-whisper': '20230314', 'opencensus': '0.11.2', 'opencensus-context': '0.1.3', 'opensearch-py': '2.2.0', 'orjson': '3.8.12', 'outcome': '1.2.0', 'packaging': '23.1', 'pandas': '2.0.1', 'pandocfilters': '1.5.0', 'parso': '0.8.3', 'passlib': '1.7.4', 'pathspec': '0.11.1', 'pdf2image': '1.16.3', 'pexpect': '4.8.0', 'pickleshare': '0.7.5', 'pinecone-client': '2.2.1', 'pip': '23.0.1', 'platformdirs': '3.4.0', 'pluggy': '1.0.0', 'posthog': '3.0.1', 'pre-commit': '3.2.2', 'prometheus-client': '0.13.1', 'prompt-toolkit': '3.0.38', 'protobuf': '3.20.2', 'psutil': '5.9.0', 'psycopg2-binary': '2.9.6', 'ptyprocess': '0.7.0', 'pure-eval': '0.2.2', 'py3nvml': '0.2.7', 'py-cpuinfo': '9.0.0', 'py-spy': '0.3.14', 'pyarrow': '11.0.0', 'pyasn1': '0.5.0', 'pyasn1-modules': '0.3.0', 'pycparser': '2.21', 'pydantic': '1.10.7', 'pydoc-markdown': '4.6.4', 'pylint': '2.17.3', 'pymilvus': '2.0.2', 'pyparsing': '2.4.7', 'pyproject-api': '1.5.1', 'pyrsistent': '0.19.3', 'pytesseract': '0.3.10', 'pytest': '7.3.1', 'pytest-asyncio': '0.21.0', 'pytest-custom-exit-code': '0.3.0', 'python-dateutil': '2.8.2', 'python-docx': '0.8.11', 'python-dotenv': '1.0.0', 'python-frontmatter': '1.0.0', 'python-jose': '3.3.0', 'python-json-logger': '2.0.7', 'python-magic': '0.4.27', 'python-multipart': '0.0.6', 'python-terraform': '0.10.1', 'pytrec-eval': '0.5', 'pytz': '2023.3', 'pyyaml-env-tag': '0.1', 'pyzmq': '25.0.2', 'quantulum3': '0.8.1', 'querystring-parser': '1.2.4', 'rank-bm25': '0.2.2', 'rapidfuzz': '2.7.0', 'ray': '1.12.1', 'rdflib': '6.3.1', 'regex': '2023.3.23', 'requests': '2.28.2', 'requests-cache': '0.9.8', 'requests-oauthlib': '1.3.1', 'responses': '0.18.0', 'rfc3339-validator': '0.1.4', 'rfc3986-validator': '0.1.1', 'rich': '12.6.0', 'rsa': '4.9', 'scikit-learn': '1.2.2', 'scipy': '1.10.1', 'selenium': '4.9.0', 'sentence-transformers': '2.2.2', 'sentencepiece': '0.1.98', 'seqeval': '1.2.2', 'setuptools': '66.0.0', 'six': '1.16.0', 'smart-open': '6.3.0', 'smmap': '5.0.0', 'sniffio': '1.3.0', 'sortedcontainers': '2.4.0', 'soupsieve': '2.4.1', 'sqlalchemy2-stubs': '0.0.2a34', 'sqlmodel': '0.0.8', 'sqlparse': '0.4.4', 'sseclient-py': '1.7.2', 'stack-data': '0.6.2', 'starlette': '0.17.1', 'sympy': '1.11.1', 'tabulate': '0.9.0', 'tenacity': '8.2.2', 'terminado': '0.17.1', 'threadpoolctl': '3.1.0', 'tika': '2.6.0', 'tiktoken': '0.3.1', 'tinycss2': '1.2.1', 'tokenize-rt': '5.0.0', 'tokenizers': '0.13.3', 'tomli': '2.0.1', 'tomli-w': '1.0.0', 'tomlkit': '0.11.8', 'torch': '2.0.0', 'torchvision': '0.15.1', 'tornado': '6.3.1', 'tox': '4.5.1', 'tqdm': '4.65.0', 'traitlets': '5.9.0', 'transformers': '4.25.1', 'trio': '0.22.0', 'trio-websocket': '0.10.2', 'triton': '2.0.0', 'typing-extensions': '4.5.0', 'tzdata': '2023.3', 'ujson': '5.1.0', 'uri-template': '1.2.0', 'url-normalize': '1.4.3', 'urllib3': '1.26.15', 'uvicorn': '0.17.6', 'uvloop': '0.17.0', 'validators': '0.18.2', 'virtualenv': '20.22.0', 'watchdog': '3.0.0', 'watchgod': '0.8.2', 'wcwidth': '0.2.6', 'weaviate-client': '3.10.0', 'webcolors': '1.13', 'webdriver-manager': '3.8.6', 'webencodings': '0.5.1', 'websocket-client': '1.5.1', 'websockets': '11.0.3', 'wheel': '0.38.4', 'widgetsnbextension': '3.6.4', 'wrapt': '1.15.0', 'wsproto': '1.2.0', 'xmltodict': '0.13.0', 'xxhash': '3.2.0', 'yapf': '0.33.0', 'yarl': '1.9.2', 'zenml': '0.38.0', 'zipp': '3.15.0'}
CURRENT STACK
Name: default ID: 569c9ea0-e6d4-414b-b0f0-1c5430dbce11 Shared: No User: default / 193c1ec4-ff4d-4dbc-9de5-11be8522f668 Workspace: default / 30a7c238-faf0-4a7c-a08b-938176350825
ORCHESTRATOR: default
Name: default ID: 8563019f-4821-43e0-b77e-73d274c59a44 Type: orchestrator Flavor: local Configuration: {} Shared: No User: default / 193c1ec4-ff4d-4dbc-9de5-11be8522f668 Workspace: default / 30a7c238-faf0-4a7c-a08b-938176350825
ARTIFACT_STORE: default
Name: default ID: 05c0c57a-ee1c-4628-b66e-a15cf7b4f692 Type: artifact_store Flavor: local Configuration: {'path': ''} Shared: No User: default / 193c1ec4-ff4d-4dbc-9de5-11be8522f668 Workspace: default / 30a7c238-faf0-4a7c-a08b-938176350825
What happened?
Scenario 2: Local Deployment of the HTTP server with docker is not working as intended.
Docker is running in rootless context.
Non docker Scenario 2: Local Deployment of the HTTP server is working as intended.
Reproduction steps
$ zenml up --docker
Relevant log output
Code of Conduct