search

iterative / dvc-ssh

SSH/SFTP plugin for dvc
Apache License 2.0
1 stars 3 forks source link

Global config for ssh jobs #35

Closed johnyaku closed 1 year ago

johnyaku commented 1 year ago

Issue #16 discusses problems caused by DVC initiating too many sessions when communicating with SSH remotes.

This can be configured per remote per project with

dvc remote modify <remote_name> jobs <limit>

The number of sessions is 4 x jobs, so setting jobs to 2 will keep us under the default value of 10 for MaxSessions in /etc/ssh/sshd_config.

It would be helpful if a default value for jobs could be set globally.

pmrowla commented 1 year ago

This is already possible using the --global or --system flags (i.e. dvc remote modify --global <remote_name> ...). The config setting will still be per remote, but can be applied to all DVC repo's that use the given remote name for either yourself (with --global) or for anyone using your machine (with --system).

See the docs for dvc config for details https://dvc.org/doc/command-reference/config#description

johnyaku commented 1 year ago

Thanks @pmrowla