[BUG]: Trying to run `zenml go` in a docker container results in errors

Contact Details [Optional]

No response

System Information

ZENML_LOCAL_VERSION: 0.43.0 ZENML_SERVER_VERSION: 0.43.0 ZENML_SERVER_DATABASE: sqlite ZENML_SERVER_DEPLOYMENT_TYPE: other ZENML_CONFIG_DIR: /root/.config/zenml ZENML_LOCAL_STORE_DIR: /root/.config/zenml/local_stores ZENML_SERVER_URL: sqlite:////root/.config/zenml/local_stores/default_zen_store/zenml.db ZENML_ACTIVE_REPOSITORY_ROOT: None PYTHON_VERSION: 3.11.4 ENVIRONMENT: native SYSTEM_INFO: {'os': 'linux', 'linux_distro': 'ubuntu', 'linux_distro_like': 'debian', 'linux_distro_version': '23.04'} ACTIVE_WORKSPACE: default ACTIVE_STACK: default ACTIVE_USER: default TELEMETRY_STATUS: disabled ANALYTICS_CLIENT_ID: 38971716-74f6-47aa-9525-332d6258675b ANALYTICS_USER_ID: e18d86b1-d795-476a-8c9a-7cd265254991 ANALYTICS_SERVER_ID: 38971716-74f6-47aa-9525-332d6258675b INTEGRATIONS: ['kaniko'] PACKAGES: {'babel': '2.12.1', 'gitpython': '3.1.32', 'jinja2': '3.1.2', 'mako': '1.2.4', 'markupsafe': '2.1.3', 'pymysql': '1.0.3', 'pyyaml': '6.0.1', 'pygments': '2.16.1', 'sqlalchemy': '1.4.41', 'sqlalchemy-utils': '0.38.3', 'send2trash': '1.8.2', 'alabaster': '0.7.13', 'alembic': '1.8.1', 'analytics-python': '1.4.post1', 'anyio': '3.7.1', 'argon2-cffi': '23.1.0', 'argon2-cffi-bindings': '21.2.0', 'arrow': '1.2.3', 'asttokens': '2.2.1', 'async-lru': '2.0.4', 'attrs': '23.1.0', 'azure-common': '1.1.28', 'azure-core': '1.29.2', 'azure-mgmt-core': '1.4.0', 'azure-mgmt-resource': '23.0.1', 'backcall': '0.2.0', 'backoff': '1.10.0', 'bcrypt': '4.0.1', 'beautifulsoup4': '4.12.2', 'black': '23.7.0', 'bleach': '6.0.0', 'boto3': '1.28.26', 'botocore': '1.31.26', 'certifi': '2023.7.22', 'cffi': '1.15.1', 'charset-normalizer': '3.2.0', 'click': '8.1.3', 'click-params': '0.3.0', 'cloudpickle': '2.2.1', 'colorama': '0.4.6', 'comm': '0.1.4', 'commonmark': '0.9.1', 'coverage': '7.3.0', 'debugpy': '1.6.7.post1', 'decorator': '5.1.1', 'defusedxml': '0.7.1', 'distro': '1.8.0', 'docker': '6.1.3', 'docutils': '0.20.1', 'executing': '1.2.0', 'fastjsonschema': '2.18.0', 'fqdn': '1.5.1', 'furo': '2023.7.26', 'gitdb': '4.0.10', 'greenlet': '2.0.2', 'httplib2': '0.19.1', 'idna': '3.4', 'imagesize': '1.4.1', 'iniconfig': '2.0.0', 'ipykernel': '6.25.1', 'ipython': '8.14.0', 'ipython-genutils': '0.2.0', 'ipywidgets': '7.8.0', 'isodate': '0.6.1', 'isoduration': '20.11.0', 'jedi': '0.19.0', 'jmespath': '1.0.1', 'json5': '0.9.14', 'jsonpointer': '2.4', 'jsonschema': '4.19.0', 'jsonschema-specifications': '2023.7.1', 'jupyter-client': '8.3.0', 'jupyter-core': '5.3.1', 'jupyter-events': '0.7.0', 'jupyter-lsp': '2.2.0', 'jupyter-server': '2.7.1', 'jupyter-server-terminals': '0.4.4', 'jupyterlab': '4.0.5', 'jupyterlab-pygments': '0.2.2', 'jupyterlab-server': '2.24.0', 'jupyterlab-widgets': '1.1.5', 'livereload': '2.6.3', 'matplotlib-inline': '0.1.6', 'mistune': '3.0.1', 'monotonic': '1.6', 'mypy': '1.5.0', 'mypy-extensions': '1.0.0', 'nbclient': '0.8.0', 'nbconvert': '7.7.3', 'nbformat': '5.9.2', 'nest-asyncio': '1.5.7', 'notebook': '7.0.2', 'notebook-shim': '0.2.3', 'numpy': '1.25.2', 'overrides': '7.4.0', 'packaging': '22.0', 'pandas': '2.0.3', 'pandocfilters': '1.5.0', 'parso': '0.8.3', 'passlib': '1.7.4', 'pathspec': '0.11.2', 'pexpect': '4.8.0', 'pickleshare': '0.7.5', 'pip': '23.2.1', 'platformdirs': '3.10.0', 'pluggy': '1.2.0', 'prometheus-client': '0.17.1', 'prompt-toolkit': '3.0.39', 'psutil': '5.9.5', 'ptyprocess': '0.7.0', 'pure-eval': '0.2.2', 'pycparser': '2.21', 'pydantic': '1.10.12', 'pyparsing': '2.4.7', 'pytest': '7.4.0', 'python-dateutil': '2.8.2', 'python-json-logger': '2.0.7', 'python-terraform': '0.10.1', 'pytz': '2023.3', 'pyzmq': '25.1.1', 'referencing': '0.30.2', 'requests': '2.31.0', 'rfc3339-validator': '0.1.4', 'rfc3986-validator': '0.1.1', 'rich': '12.6.0', 'rpds-py': '0.9.2', 'ruff': '0.0.284', 's3transfer': '0.6.1', 'setuptools': '68.0.0', 'six': '1.16.0', 'smmap': '5.0.0', 'sniffio': '1.3.0', 'snowballstemmer': '2.2.0', 'soupsieve': '2.4.1', 'sphinx': '7.1.2', 'sphinx-autobuild': '2021.3.14', 'sphinx-basic-ng': '1.0.0b2', 'sphinxcontrib-applehelp': '1.0.7', 'sphinxcontrib-devhelp': '1.0.5', 'sphinxcontrib-htmlhelp': '2.0.4', 'sphinxcontrib-jsmath': '1.0.1', 'sphinxcontrib-qthelp': '1.0.6', 'sphinxcontrib-serializinghtml': '1.1.8', 'sqlalchemy2-stubs': '0.0.2a35', 'sqlmodel': '0.0.8', 'stack-data': '0.6.2', 'terminado': '0.17.1', 'tinycss2': '1.2.1', 'tornado': '6.3.3', 'traitlets': '5.9.0', 'typing-extensions': '4.7.1', 'tzdata': '2023.3', 'uri-template': '1.3.0', 'urllib3': '1.26.16', 'validators': '0.18.2', 'wcwidth': '0.2.6', 'webcolors': '1.13', 'webencodings': '0.5.1', 'websocket-client': '1.6.1', 'wf-aws': '1.12.2', 'wf-blackbox-zero': '1.1.0', 'wf-cli': '1.25.0', 'wf-cli-plugin-aws': '1.5.6', 'wf-cli-plugin-docs': '1.2.6', 'wf-cli-plugin-login': '1.0.5', 'wf-cli-plugin-py': '2.3.4', 'wf-zenml-pipelines': '0.0.0', 'widgetsnbextension': '3.6.5', 'zenml': '0.43.0'}

CURRENT STACK

Name: default ID: c5036656-d46b-4988-8b85-5cc6221eb0eb Shared: No User: default / e18d86b1-d795-476a-8c9a-7cd265254991 Workspace: default / 6d5238df-adb2-4d00-8804-856fbecfcfbf

ORCHESTRATOR: default

Name: default ID: cf9ab169-8724-4602-b56f-9bbbb830d0ca Type: orchestrator Flavor: local Configuration: {} Shared: No User: default / e18d86b1-d795-476a-8c9a-7cd265254991 Workspace: default / 6d5238df-adb2-4d00-8804-856fbecfcfbf

ARTIFACT_STORE: default

Name: default ID: 22fab2ea-f53a-4d49-95c6-7800c532b994 Type: artifact_store Flavor: local Configuration: {'path': ''} Shared: No User: default / e18d86b1-d795-476a-8c9a-7cd265254991 Workspace: default / 6d5238df-adb2-4d00-8804-856fbecfcfbf

What happened?

Trying to run the zenml tutorial on a dev container (docker), when it gets to the point of running the jupyter notebook server the process fails because commands in the docker container are run as root.

This can be solved in a variety of ways:

Manually running the jupyter notebook server and opening the tutorial notebooks from there
Allowing zenml to pass the --allow-root flag when instantiating the jupyter notebook server (for instance, if some env variable is present)
Not launching the jupyter notebook server at all, and instead using vscode notebook extension to explore the tutorial notebooks

However, the zenml go path does not suggest any ways of fixing this, instead crashes with the very cryptic error message

CalledProcessError: Command '['jupyter', 'notebook']' returned non-zero exit status 1.

and a really long non-informative stack trace

Reproduction steps

Make a docker container that contains zenml as a dependency (eg a dev container)
In the container shell, run zenml go ad go through the steps

This error will come up:

I 2023-08-15 15:46:20.910 ServerApp] jupyterlab | extension was successfully loaded.
[I 2023-08-15 15:46:20.911 ServerApp] notebook | extension was successfully loaded.
[C 2023-08-15 15:46:20.912 ServerApp] Running as root is not recommended. Use --allow-root to bypass.

Relevant log output

🧑‍🏫 Get started with ZenML                           

The ZenML tutorial repository was cloned to your current working directory.     
Within the repository you can get started on one of these notebooks:            

 • /workspaces/zenml_pipelines/zenml_tutorial/notebooks/quickstart-zenml-cloud.i
   pynb                                                                         
 • /workspaces/zenml_pipelines/zenml_tutorial/notebooks/quickstart.ipynb        

Next we will start a Jupyter notebook server. Feel free to try your hand at our 
tutorial notebooks. If your browser does not open automatically click one of the
links below.                                                                    
Press ENTER to continue...
[I 2023-08-15 15:46:20.494 ServerApp] Package notebook took 0.0000s to import
[I 2023-08-15 15:46:20.508 ServerApp] Package jupyter_lsp took 0.0135s to import
[W 2023-08-15 15:46:20.508 ServerApp] A `_jupyter_server_extension_points` function was not found in jupyter_lsp. Instead, a `_jupyter_server_extension_paths` function was found and will be used for now. This function name will be deprecated in future releases of Jupyter Server.
[I 2023-08-15 15:46:20.515 ServerApp] Package jupyter_server_terminals took 0.0070s to import
[I 2023-08-15 15:46:20.515 ServerApp] Package jupyterlab took 0.0000s to import
[I 2023-08-15 15:46:20.771 ServerApp] Package notebook_shim took 0.0000s to import
[W 2023-08-15 15:46:20.771 ServerApp] A `_jupyter_server_extension_points` function was not found in notebook_shim. Instead, a `_jupyter_server_extension_paths` function was found and will be used for now. This function name will be deprecated in future releases of Jupyter Server.
[I 2023-08-15 15:46:20.772 ServerApp] jupyter_lsp | extension was successfully linked.
[I 2023-08-15 15:46:20.774 ServerApp] jupyter_server_terminals | extension was successfully linked.
[I 2023-08-15 15:46:20.776 ServerApp] jupyterlab | extension was successfully linked.
[I 2023-08-15 15:46:20.778 ServerApp] notebook | extension was successfully linked.
[I 2023-08-15 15:46:20.779 ServerApp] Writing Jupyter server cookie secret to /opt/wf-state/.local/share/jupyter/runtime/jupyter_cookie_secret
[I 2023-08-15 15:46:20.886 ServerApp] notebook_shim | extension was successfully linked.
[I 2023-08-15 15:46:20.906 ServerApp] notebook_shim | extension was successfully loaded.
[I 2023-08-15 15:46:20.907 ServerApp] jupyter_lsp | extension was successfully loaded.
[I 2023-08-15 15:46:20.908 ServerApp] jupyter_server_terminals | extension was successfully loaded.
[I 2023-08-15 15:46:20.909 LabApp] JupyterLab extension loaded from /workspaces/zenml_pipelines/.venv/lib/python3.11/site-packages/jupyterlab
[I 2023-08-15 15:46:20.909 LabApp] JupyterLab application directory is /workspaces/zenml_pipelines/.venv/share/jupyter/lab
[I 2023-08-15 15:46:20.909 LabApp] Extension Manager is 'pypi'.
[I 2023-08-15 15:46:20.910 ServerApp] jupyterlab | extension was successfully loaded.
[I 2023-08-15 15:46:20.911 ServerApp] notebook | extension was successfully loaded.
[C 2023-08-15 15:46:20.912 ServerApp] Running as root is not recommended. Use --allow-root to bypass.
╭─────────────────────────────── Traceback (most recent call last) ────────────────────────────────╮
│ /workspaces/zenml_pipelines/.venv/bin/zenml:8 in <module>                                        │
│                                                                                                  │
│   5 from zenml.cli.cli import cli                                                                │
│   6 if __name__ == "__main__":                                                                   │
│   7 │   sys.argv[0] = re.sub(r"(-script\.pyw|\.exe)?$", "", sys.argv[0])                         │
│ ❱ 8 │   sys.exit(cli())                                                                          │
│   9                                                                                              │
│                                                                                                  │
│ /workspaces/zenml_pipelines/.venv/lib/python3.11/site-packages/click/core.py:1130 in __call__    │
│                                                                                                  │
│   1127 │                                                                                         │
│   1128 │   def __call__(self, *args: t.Any, **kwargs: t.Any) -> t.Any:                           │
│   1129 │   │   """Alias for :meth:`main`."""                                                     │
│ ❱ 1130 │   │   return self.main(*args, **kwargs)                                                 │
│   1131                                                                                           │
│   1132                                                                                           │
│   1133 class Command(BaseCommand):                                                               │
│                                                                                                  │
│ /workspaces/zenml_pipelines/.venv/lib/python3.11/site-packages/click/core.py:1055 in main        │
│                                                                                                  │
│   1052 │   │   try:                                                                              │
│   1053 │   │   │   try:                                                                          │
│   1054 │   │   │   │   with self.make_context(prog_name, args, **extra) as ctx:                  │
│ ❱ 1055 │   │   │   │   │   rv = self.invoke(ctx)                                                 │
│   1056 │   │   │   │   │   if not standalone_mode:                                               │
│   1057 │   │   │   │   │   │   return rv                                                         │
│   1058 │   │   │   │   │   # it's not safe to `ctx.exit(rv)` here!                               │
│                                                                                                  │
│ /workspaces/zenml_pipelines/.venv/lib/python3.11/site-packages/click/core.py:1657 in invoke      │
│                                                                                                  │
│   1654 │   │   │   │   super().invoke(ctx)                                                       │
│   1655 │   │   │   │   sub_ctx = cmd.make_context(cmd_name, args, parent=ctx)                    │
│   1656 │   │   │   │   with sub_ctx:                                                             │
│ ❱ 1657 │   │   │   │   │   return _process_result(sub_ctx.command.invoke(sub_ctx))               │
│   1658 │   │                                                                                     │
│   1659 │   │   # In chain mode we create the contexts step by step, but after the                │
│   1660 │   │   # base command has been invoked.  Because at that point we do not                 │
│                                                                                                  │
│ /workspaces/zenml_pipelines/.venv/lib/python3.11/site-packages/click/core.py:1404 in invoke      │
│                                                                                                  │
│   1401 │   │   │   echo(style(message, fg="red"), err=True)                                      │
│   1402 │   │                                                                                     │
│   1403 │   │   if self.callback is not None:                                                     │
│ ❱ 1404 │   │   │   return ctx.invoke(self.callback, **ctx.params)                                │
│   1405 │                                                                                         │
│   1406 │   def shell_complete(self, ctx: Context, incomplete: str) -> t.List["CompletionItem"]:  │
│   1407 │   │   """Return a list of completions for the incomplete value. Looks                   │
│                                                                                                  │
│ /workspaces/zenml_pipelines/.venv/lib/python3.11/site-packages/click/core.py:760 in invoke       │
│                                                                                                  │
│    757 │   │                                                                                     │
│    758 │   │   with augment_usage_errors(__self):                                                │
│    759 │   │   │   with ctx:                                                                     │
│ ❱  760 │   │   │   │   return __callback(*args, **kwargs)                                        │
│    761 │                                                                                         │
│    762 │   def forward(                                                                          │
│    763 │   │   __self, __cmd: "Command", *args: t.Any, **kwargs: t.Any  # noqa: B902             │
│                                                                                                  │
│ /workspaces/zenml_pipelines/.venv/lib/python3.11/site-packages/zenml/cli/base.py:405 in go       │
│                                                                                                  │
│   402 │   │   )                                                                                  │
│   403 │   │   input("Press ENTER to continue...")                                                │
│   404 │   notebook_path = os.path.join(zenml_tutorial_path, "notebooks")                         │
│ ❱ 405 │   subprocess.check_call(["jupyter", "notebook"], cwd=notebook_path)                      │
│   406                                                                                            │
│   407                                                                                            │
│   408 def _prompt_email(event_source: AnalyticsEventSource) -> bool:                             │
│                                                                                                  │
│ /usr/lib/python3.11/subprocess.py:413 in check_call                                              │
│                                                                                                  │
│    410 │   │   cmd = kwargs.get("args")                                                          │
│    411 │   │   if cmd is None:                                                                   │
│    412 │   │   │   cmd = popenargs[0]                                                            │
│ ❱  413 │   │   raise CalledProcessError(retcode, cmd)                                            │
│    414 │   return 0                                                                              │
│    415                                                                                           │
│    416                                                                                           │
╰──────────────────────────────────────────────────────────────────────────────────────────────────╯
CalledProcessError: Command '['jupyter', 'notebook']' returned non-zero exit status 1.

Code of Conduct

[X] I agree to follow this project's Code of Conduct

zenml-io / zenml