By default, git commands run by GitInput.pm time out after 600 seconds. This is not enough time to perform the first fetch of nixpkgs (2.19GiB) on a VDSL (~15mbps) connection.
Additionally, the timeout doesn't seem to be documented outside of the GitInput.pm file itself, and it is hard to find from the error message (which is just the word "timeout") creating a pretty poor experience for first-time users.
Even on my gigabit connection, I only get 80mbps from GitHub*, which means a fresh nixpkgs clone takes 329.71s wall time to transfer the necessary 2.19GiB. This provides very little wiggle room for nixpkgs to get larger or for GitHub to get slower.
*Not sure if this is a universal experience, or if GitHub just has poor geo-distribution and it's only an issue for non-US people.
Happy to PR bumping the timeout (3600s?) if that's an acceptable approach. An analogue of nix's max-silent-time might be a more elegant solution if the goal is to detect hanging/stuck checkouts, though.
To Reproduce
Steps to reproduce the behavior:
Limit internet speed to 20mbps, e.g. tc qdisc add dev eth0 root tbf rate 20000kbit burst 20000
Create a jobset with a git input of nixpkgs
Observe that evaluation fails
Logs
Mar 23 08:06:48 nixos hydra-evaluator[79757]: error fetching latest change from git repo at `https://github.com/nixos/nixpkgs.git':
Mar 23 08:06:48 nixos hydra-evaluator[79757]: timeout
Hydra Server:
OS and version: NixOS 21.11.336619.01e87b327aa (Porcupine)
Version of Hydra: nixpkgs's hydra-unstable 2021-08-11
Describe the bug
By default, git commands run by
GitInput.pm
time out after 600 seconds. This is not enough time to perform the first fetch of nixpkgs (2.19GiB) on a VDSL (~15mbps) connection.Additionally, the timeout doesn't seem to be documented outside of the GitInput.pm file itself, and it is hard to find from the error message (which is just the word "timeout") creating a pretty poor experience for first-time users.
Even on my gigabit connection, I only get 80mbps from GitHub*, which means a fresh nixpkgs clone takes 329.71s wall time to transfer the necessary 2.19GiB. This provides very little wiggle room for nixpkgs to get larger or for GitHub to get slower.
*Not sure if this is a universal experience, or if GitHub just has poor geo-distribution and it's only an issue for non-US people.
https://github.com/NixOS/hydra/blob/01fb23ddf647ee7b6dc3abc1f012eaf9f5306c88/src/lib/Hydra/Plugin/GitInput.pm#L80-L83
Happy to PR bumping the timeout (3600s?) if that's an acceptable approach. An analogue of nix's
max-silent-time
might be a more elegant solution if the goal is to detect hanging/stuck checkouts, though.To Reproduce Steps to reproduce the behavior:
tc qdisc add dev eth0 root tbf rate 20000kbit burst 20000
Logs
Hydra Server: