NixOS / nix

Nix, the purely functional package manager
https://nixos.org/
GNU Lesser General Public License v2.1
12.95k stars 1.53k forks source link

Keep connection to remote builder alive to improve remote build latency #8499

Open roberth opened 1 year ago

roberth commented 1 year ago

Is your feature request related to a problem? Please describe.

Currently when doing remote builds, I see a lot of connecting to <hostname>, which can be painfully slow when the remote is in another network, or when the builds are serialized, such as when doing IFD. Deep changes to config files, such as a small configuration change in a NixOS generation may also be particularly affected by this latency, unless we use or advance knowledge about the dependency graph in that case (don't think we do, but also not sure).

Describe the solution you'd like

Have a connection pool for each of the builders.

Describe alternatives you've considered

Use an SSH master connection instead of a pool. I don't think we should force use of this feature though; it is more fragile and it complicates the user's configurations.

Additional context

Priorities

Add :+1: to issues you find important.

teto commented 2 weeks ago

this hits me especially strongly this morning. While a native solution would be better, maybe it can be done outside nix with a correct ssh configuration ? something like

  controlmaster auto
  controlpath /tmp/ssh-%r@%h:%p
grahamc commented 2 weeks ago

This is what hydra and similar systems use 👍

On Tue, Nov 19, 2024, at 5:32 AM, Matthieu Coudron wrote:

this hits me especially strongly this morning. While a native solution would be better, maybe it can be done outside nix with a correct ssh configuration ? something like

controlmaster auto controlpath /tmp/ssh-%r@%h:%p

— Reply to this email directly, view it on GitHub https://github.com/NixOS/nix/issues/8499#issuecomment-2485308637, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAASXLHFRVR4ATP7Y74S5V32BMHTHAVCNFSM6AAAAABSBYI42GVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDIOBVGMYDQNRTG4. You are receiving this because you are subscribed to this thread.Message ID: @.***>