dstackai / dstack

dstack is an easy-to-use and flexible container orchestrator for running AI workloads in any cloud or data center.
https://dstack.ai
Mozilla Public License 2.0
1.24k stars 91 forks source link

[Bug] `dstack pool add-ssh` needs `sudo` access #1381

Open peterschmidt85 opened 2 weeks ago

peterschmidt85 commented 2 weeks ago

Steps to reproduce:

  1. Set up a host where the user requires a password for sudo
  2. Invoke dstack pool add-ssh

Actual behavior:

The server logs:

{"message": "Try to connect to dex@10.15.105.45:22 with key SHA256:wIVG0/gEoe+KQfvoCUH7GEZqG6mvtLSirjqqkyqiPTo", "logger": "dstack._internal.core.backends.remote.provisioning", "timestamp": "2024-07-01 23:07:45,550", "level": "DEBUG"}
{"message": "Connected to dex 10.15.105.45", "logger": "dstack._internal.server.background.tasks.process_instances", "timestamp": "2024-07-01 23:07:46,238", "level": "INFO"}
{"message": "deploy_instance raise an error: The command 'run_pre_start_commands' didn't work. stdout: , stderr: sudo: a terminal is required to read the password; either use the -S option to read from standard input or configure an askpass helper\nsudo: a password is required", "logger": "dstack._internal.server.background.tasks.process_instances", "timestamp": "2024-07-01 23:07:46,823", "level": "DEBUG"}
{"message": "Provisioning the instance 'hungry-rat' could not be completed because of the error: Deploy instance raise an error The command 'run_pre_start_commands' didn't work. stdout: , stderr: sudo: a terminal is required to read the password; either use the -S option to read from standard input or configure an askpass helper\nsudo: a password is required", "logger": "dstack._internal.server.background.tasks.process_instances", "timestamp": "2024-07-01 23:07:46,824", "level": "WARNING"}

Notes:

Currently, dstack pool add-ssh requires sudo access (e.g. because it needs access to systemd to create/update the dstack-shim service).

Workaround:

  1. Create a user that doesn't require password for sudo and use with dstack pool add-ssh
  2. Configure the existing user not to ask the password for sudo (example)

Also, ensure the user is in the sudo group

r4victor commented 2 weeks ago

The problem takes place when using a sudo user that is configured to ask for a password. Usually this doesn't happen on cloud instances because cloud-init by default sets up ubuntu user with no password sudo.

An alternative workaround to using root is to configure a non-root sudo user without password.