search

hercules-ci / hercules-ci-agent

https://hercules-ci.com build and deployment agent
Apache License 2.0
99 stars 19 forks source link

hercules-ci-nix-daemon processes leftover after effect task completes #449

Open max-privatevoid opened 2 years ago

max-privatevoid commented 2 years ago

Description On a non-NixOS machine, with Nix installed in single user mode and Hercules CI Agent installed in .nix-profile, executed via a systemd user service, there seem to be hercules-ci-nix-daemon processes sticking around indefinitely after effect tasks are completed. This problem may also exist on NixOS, but may be less noticeable thanks to hercules-ci-agent-restarter.

The effects in question are two runNixOS effects from https://github.com/privatevoid-net/privatevoid-infrastructure

To Reproduce Can't reliably reproduce.

Expected behavior No leftover hercules-ci-nix-daemon processes.

Logs There seem to be no relevant logs, but this is what the processes look like.

● hercules-ci-agent.service
     Loaded: loaded (/var/lib/nixbuilder/.config/systemd/user/hercules-ci-agent.service; static)
     Active: active (running) since Tue 2022-09-13 21:57:33 CEST; 1 weeks 0 days ago
    Process: 3470412 ExecStartPre=/var/lib/nixbuilder/.nix-profile/bin/hercules-ci-agent --config /var/lib/nixbuilder/.config/hercules-ci/agent.toml --test-configuration (code=exited, status=0/SUCCESS)
   Main PID: 3470435 (.hercules-ci-ag)
     CGroup: /user.slice/user-984.slice/user@984.service/hercules-ci-agent.service
             ├─1818684 /nix/store/jpjcbqcad6m0f833j0crsixlj9v4jfrw-hercules-ci-agent-0.9.7/bin/hercules-ci-nix-daemon --option netrc-file /var/lib/nixbuilder/.cache/hercules-ci/work/secure/tmp-netrc3470435-837.key --option substituters s3://nix-store?endpoint=object-storage.privatevoid.net https://cache.nixos.org/ https://cache.privatevoid.net/ https://max.cachix.org?priority=60 --option trusted-public-keys cache.privatevoid.net:SErQ8bvNWANeAvtsOESUwVYr2VJynfuc9JRwlzTTkVg= cache.nixos.org-1:6NCHdD59X431o0gWypbMrAURkbJ16ZPMQFGspcDShjY= cache.privatevoid.net:SErQ8bvNWANeAvtsOESUwVYr2VJynfuc9JRwlzTTkVg= max.cachix.org-1:oSMQ1zYLR8H4L17hfe6ETlI/d+VeiBykB8PbBdPtDJw= --option narinfo-cache-negative-ttl 0 --option max-jobs 1
             ├─1818685 /nix/store/jpjcbqcad6m0f833j0crsixlj9v4jfrw-hercules-ci-agent-0.9.7/bin/hercules-ci-nix-daemon --option netrc-file /var/lib/nixbuilder/.cache/hercules-ci/work/secure/tmp-netrc3470435-836.key --option substituters s3://nix-store?endpoint=object-storage.privatevoid.net https://cache.nixos.org/ https://cache.privatevoid.net/ https://max.cachix.org?priority=60 --option trusted-public-keys cache.privatevoid.net:SErQ8bvNWANeAvtsOESUwVYr2VJynfuc9JRwlzTTkVg= cache.nixos.org-1:6NCHdD59X431o0gWypbMrAURkbJ16ZPMQFGspcDShjY= cache.privatevoid.net:SErQ8bvNWANeAvtsOESUwVYr2VJynfuc9JRwlzTTkVg= max.cachix.org-1:oSMQ1zYLR8H4L17hfe6ETlI/d+VeiBykB8PbBdPtDJw= --option narinfo-cache-negative-ttl 0 --option max-jobs 1
             ├─2575176 /nix/store/jpjcbqcad6m0f833j0crsixlj9v4jfrw-hercules-ci-agent-0.9.7/bin/hercules-ci-nix-daemon --option netrc-file /var/lib/nixbuilder/.cache/hercules-ci/work/secure/tmp-netrc3470435-885.key --option substituters s3://nix-store?endpoint=object-storage.privatevoid.net https://cache.nixos.org/ https://cache.privatevoid.net/ https://max.cachix.org?priority=60 --option trusted-public-keys cache.privatevoid.net:SErQ8bvNWANeAvtsOESUwVYr2VJynfuc9JRwlzTTkVg= cache.nixos.org-1:6NCHdD59X431o0gWypbMrAURkbJ16ZPMQFGspcDShjY= cache.privatevoid.net:SErQ8bvNWANeAvtsOESUwVYr2VJynfuc9JRwlzTTkVg= max.cachix.org-1:oSMQ1zYLR8H4L17hfe6ETlI/d+VeiBykB8PbBdPtDJw= --option narinfo-cache-negative-ttl 0 --option max-jobs 1
             ├─2575250 /nix/store/jpjcbqcad6m0f833j0crsixlj9v4jfrw-hercules-ci-agent-0.9.7/bin/hercules-ci-nix-daemon --option netrc-file /var/lib/nixbuilder/.cache/hercules-ci/work/secure/tmp-netrc3470435-886.key --option substituters s3://nix-store?endpoint=object-storage.privatevoid.net https://cache.nixos.org/ https://cache.privatevoid.net/ https://max.cachix.org?priority=60 --option trusted-public-keys cache.privatevoid.net:SErQ8bvNWANeAvtsOESUwVYr2VJynfuc9JRwlzTTkVg= cache.nixos.org-1:6NCHdD59X431o0gWypbMrAURkbJ16ZPMQFGspcDShjY= cache.privatevoid.net:SErQ8bvNWANeAvtsOESUwVYr2VJynfuc9JRwlzTTkVg= max.cachix.org-1:oSMQ1zYLR8H4L17hfe6ETlI/d+VeiBykB8PbBdPtDJw= --option narinfo-cache-negative-ttl 0 --option max-jobs 1
             ├─2641075 /nix/store/jpjcbqcad6m0f833j0crsixlj9v4jfrw-hercules-ci-agent-0.9.7/bin/hercules-ci-nix-daemon --option netrc-file /var/lib/nixbuilder/.cache/hercules-ci/work/secure/tmp-netrc3470435-905.key --option substituters s3://nix-store?endpoint=object-storage.privatevoid.net https://cache.nixos.org/ https://cache.privatevoid.net/ https://max.cachix.org?priority=60 --option trusted-public-keys cache.privatevoid.net:SErQ8bvNWANeAvtsOESUwVYr2VJynfuc9JRwlzTTkVg= cache.nixos.org-1:6NCHdD59X431o0gWypbMrAURkbJ16ZPMQFGspcDShjY= cache.privatevoid.net:SErQ8bvNWANeAvtsOESUwVYr2VJynfuc9JRwlzTTkVg= max.cachix.org-1:oSMQ1zYLR8H4L17hfe6ETlI/d+VeiBykB8PbBdPtDJw= --option narinfo-cache-negative-ttl 0 --option max-jobs 1
             ├─2641089 /nix/store/jpjcbqcad6m0f833j0crsixlj9v4jfrw-hercules-ci-agent-0.9.7/bin/hercules-ci-nix-daemon --option netrc-file /var/lib/nixbuilder/.cache/hercules-ci/work/secure/tmp-netrc3470435-906.key --option substituters s3://nix-store?endpoint=object-storage.privatevoid.net https://cache.nixos.org/ https://cache.privatevoid.net/ https://max.cachix.org?priority=60 --option trusted-public-keys cache.privatevoid.net:SErQ8bvNWANeAvtsOESUwVYr2VJynfuc9JRwlzTTkVg= cache.nixos.org-1:6NCHdD59X431o0gWypbMrAURkbJ16ZPMQFGspcDShjY= cache.privatevoid.net:SErQ8bvNWANeAvtsOESUwVYr2VJynfuc9JRwlzTTkVg= max.cachix.org-1:oSMQ1zYLR8H4L17hfe6ETlI/d+VeiBykB8PbBdPtDJw= --option narinfo-cache-negative-ttl 0 --option max-jobs 1
             ├─2644819 /nix/store/jpjcbqcad6m0f833j0crsixlj9v4jfrw-hercules-ci-agent-0.9.7/bin/hercules-ci-nix-daemon --option netrc-file /var/lib/nixbuilder/.cache/hercules-ci/work/secure/tmp-netrc3470435-920.key --option substituters s3://nix-store?endpoint=object-storage.privatevoid.net https://cache.nixos.org/ https://cache.privatevoid.net/ https://max.cachix.org?priority=60 --option trusted-public-keys cache.privatevoid.net:SErQ8bvNWANeAvtsOESUwVYr2VJynfuc9JRwlzTTkVg= cache.nixos.org-1:6NCHdD59X431o0gWypbMrAURkbJ16ZPMQFGspcDShjY= cache.privatevoid.net:SErQ8bvNWANeAvtsOESUwVYr2VJynfuc9JRwlzTTkVg= max.cachix.org-1:oSMQ1zYLR8H4L17hfe6ETlI/d+VeiBykB8PbBdPtDJw= --option narinfo-cache-negative-ttl 0 --option max-jobs 1
             ├─2644846 /nix/store/jpjcbqcad6m0f833j0crsixlj9v4jfrw-hercules-ci-agent-0.9.7/bin/hercules-ci-nix-daemon --option netrc-file /var/lib/nixbuilder/.cache/hercules-ci/work/secure/tmp-netrc3470435-919.key --option substituters s3://nix-store?endpoint=object-storage.privatevoid.net https://cache.nixos.org/ https://cache.privatevoid.net/ https://max.cachix.org?priority=60 --option trusted-public-keys cache.privatevoid.net:SErQ8bvNWANeAvtsOESUwVYr2VJynfuc9JRwlzTTkVg= cache.nixos.org-1:6NCHdD59X431o0gWypbMrAURkbJ16ZPMQFGspcDShjY= cache.privatevoid.net:SErQ8bvNWANeAvtsOESUwVYr2VJynfuc9JRwlzTTkVg= max.cachix.org-1:oSMQ1zYLR8H4L17hfe6ETlI/d+VeiBykB8PbBdPtDJw= --option narinfo-cache-negative-ttl 0 --option max-jobs 1
             ├─3470435 /var/lib/nixbuilder/.nix-profile/bin/hercules-ci-agent --config /var/lib/nixbuilder/.config/hercules-ci/agent.toml
             ├─3580436 /nix/store/jpjcbqcad6m0f833j0crsixlj9v4jfrw-hercules-ci-agent-0.9.7/bin/hercules-ci-nix-daemon --option netrc-file /var/lib/nixbuilder/.cache/hercules-ci/work/secure/tmp-netrc3470435-19.key --option substituters s3://nix-store?endpoint=object-storage.privatevoid.net https://cache.nixos.org/ https://cache.privatevoid.net/ https://max.cachix.org?priority=60 --option trusted-public-keys cache.privatevoid.net:SErQ8bvNWANeAvtsOESUwVYr2VJynfuc9JRwlzTTkVg= cache.nixos.org-1:6NCHdD59X431o0gWypbMrAURkbJ16ZPMQFGspcDShjY= cache.privatevoid.net:SErQ8bvNWANeAvtsOESUwVYr2VJynfuc9JRwlzTTkVg= max.cachix.org-1:oSMQ1zYLR8H4L17hfe6ETlI/d+VeiBykB8PbBdPtDJw= --option narinfo-cache-negative-ttl 0 --option max-jobs 1
             ├─3580521 /nix/store/jpjcbqcad6m0f833j0crsixlj9v4jfrw-hercules-ci-agent-0.9.7/bin/hercules-ci-nix-daemon --option netrc-file /var/lib/nixbuilder/.cache/hercules-ci/work/secure/tmp-netrc3470435-20.key --option substituters s3://nix-store?endpoint=object-storage.privatevoid.net https://cache.nixos.org/ https://cache.privatevoid.net/ https://max.cachix.org?priority=60 --option trusted-public-keys cache.privatevoid.net:SErQ8bvNWANeAvtsOESUwVYr2VJynfuc9JRwlzTTkVg= cache.nixos.org-1:6NCHdD59X431o0gWypbMrAURkbJ16ZPMQFGspcDShjY= cache.privatevoid.net:SErQ8bvNWANeAvtsOESUwVYr2VJynfuc9JRwlzTTkVg= max.cachix.org-1:oSMQ1zYLR8H4L17hfe6ETlI/d+VeiBykB8PbBdPtDJw= --option narinfo-cache-negative-ttl 0 --option max-jobs 1
             ├─3744459 /nix/store/jpjcbqcad6m0f833j0crsixlj9v4jfrw-hercules-ci-agent-0.9.7/bin/hercules-ci-nix-daemon --option netrc-file /var/lib/nixbuilder/.cache/hercules-ci/work/secure/tmp-netrc3470435-107.key --option substituters s3://nix-store?endpoint=object-storage.privatevoid.net https://cache.nixos.org/ https://cache.privatevoid.net/ https://max.cachix.org?priority=60 --option trusted-public-keys cache.privatevoid.net:SErQ8bvNWANeAvtsOESUwVYr2VJynfuc9JRwlzTTkVg= cache.nixos.org-1:6NCHdD59X431o0gWypbMrAURkbJ16ZPMQFGspcDShjY= cache.privatevoid.net:SErQ8bvNWANeAvtsOESUwVYr2VJynfuc9JRwlzTTkVg= max.cachix.org-1:oSMQ1zYLR8H4L17hfe6ETlI/d+VeiBykB8PbBdPtDJw= --option narinfo-cache-negative-ttl 0 --option max-jobs 1
             └─3744547 /nix/store/jpjcbqcad6m0f833j0crsixlj9v4jfrw-hercules-ci-agent-0.9.7/bin/hercules-ci-nix-daemon --option netrc-file /var/lib/nixbuilder/.cache/hercules-ci/work/secure/tmp-netrc3470435-108.key --option substituters s3://nix-store?endpoint=object-storage.privatevoid.net https://cache.nixos.org/ https://cache.privatevoid.net/ https://max.cachix.org?priority=60 --option trusted-public-keys cache.privatevoid.net:SErQ8bvNWANeAvtsOESUwVYr2VJynfuc9JRwlzTTkVg= cache.nixos.org-1:6NCHdD59X431o0gWypbMrAURkbJ16ZPMQFGspcDShjY= cache.privatevoid.net:SErQ8bvNWANeAvtsOESUwVYr2VJynfuc9JRwlzTTkVg= max.cachix.org-1:oSMQ1zYLR8H4L17hfe6ETlI/d+VeiBykB8PbBdPtDJw= --option narinfo-cache-negative-ttl 0 --option max-jobs 1

Platform / Version 0.9.7

roberth commented 2 years ago

Found two problems