Open RaitoBezarius opened 8 months ago
I realized that .drv
are probably going to stick around in the Nix store because we put GC roots on them. There is also a need to consume the .drv
, unlink them and remove them explicitly from the store, don't let the GC procedure do it, it will be too slow and resource consuming.
This way, we ingest what we need out of those .drv
and keep a reasonable operating Nix store size.
Now, we deployed the evaluation in production on spinning rust, I realize that the bottleneck is the nix-daemon performing some R/W, various metrics can be found here: http://sectracker.nixpkgs.lahfa.xyz:19999/#menu_systemd_web-security-tracker-worker;after=1703091894000;before=1703097151000;theme=slate;utc=Europe/Paris.
This is here to collect all the concerns about the long term performance of the security tracker.
A big part of the concern is with the disk space consumption rate and the query performance evolution over time.
Ideally, what we could do is maintain only a working set of data and archive old pages and freeze them so that we always work on one year or 3 months or 6 months worth of data to avoid the infinite growth of our dataset.