nixified-ai / flake

A Nix flake for many AI projects
GNU Affero General Public License v3.0
675 stars 73 forks source link

Eradicate cudaPackages.cudatoolkit from the closures #75

Open SomeoneSerge opened 11 months ago

SomeoneSerge commented 11 months ago

The cudaPackages.cudatoolkit attribute in nixpkgs is being deprecated (too slowly). This attribute corresponds to the runfile-based installation of the cuda toolkit, comes in just two huge outputs, which in addition have unreasonably many dependencies (like X, or gstreamer, or python2 even). Nixpkgs' CUDA-accelerated applications are being rewritten to use individual components of the toolkit, like buildInputs = with cudaPackages; [ cuda_cudart.dev cuda_cudart.lib cuda_cudart.static libcublas.dev libcublas.static ] (the example is a bit tedious atm, cc https://github.com/NixOS/nixpkgs/issues/271792), etc. One benefit of doing so, aside from better cache reuse, is that most of the cuda inputs (e.g. libcublas.static, which is huge) are automatically discarded by Nix after the build as long as they're not referenced from the application package's outputs (naturally, the static archives usually aren't) so even if the build requires tens of gigabytes of storage, the runtime closure can as small as 3GIB.