search

NixOS / nix

Nix, the purely functional package manager
https://nixos.org/
GNU Lesser General Public License v2.1
11.51k stars 1.44k forks source link

Support garbage collecting based on inodes #5508

Open grahamc opened 2 years ago

grahamc commented 2 years ago

Is your feature request related to a problem? Please describe.

Filesystems like ext4 have a pre-allocated number of inodes. Sometimes if a Nix machine builds lots of small files it will run out of inodes before it gets close to running out of space. Nix's derivations exacerbate this problem as well.

Nix's garbage collector has a way to request a given amount of disk space to , be freed, but it would be useful to also request a given number of inodes be freed.

This is especially a problem on a busy build machines like ofborg and hydra evaulators.

Describe the solution you'd like

$ nix-collect-garbage --max-freed 42560675840 --free-inodes 178163

Ideally both conditions would be met. First the garbage collector would proceed as it does today, counting the number of files and directories it deletes. Then, it would garbage collect .drv files and other small store paths until it has deleted 178163 files.

Describe alternatives you've considered

In the past I have bodged together things like find and xargs and nix-store --delete to free small files.

risicle commented 1 year ago

I've been playing with some ideas around "smarter" gc strategies over @ https://github.com/risicle/nix-heuristic-gc, which includes some options around targeting inode-heavy paths.