search

NixOS / nixfmt

The official (but not yet stable) formatter for Nix code
https://nixfmt.serokell.io
Mozilla Public License 2.0
856 stars 33 forks source link

Multithreading and performance optimization #211

Open oxalica opened 3 months ago

oxalica commented 3 months ago

It seems that we are only using a single thread when formatting tons of files. But this can be trivially parallelized. This should also ease the adoption of format-checking in git commit hook as mentioned in https://github.com/NixOS/nixpkgs/pull/322537#issuecomment-2196405557

Eg: fd -e nix --exec-batch nixfmt (--exec-batch passes as much files to nixfmt as possible in a single exec) It takes takes 1:49.27 (~110s) on my machine with only a single thread (~99% CPU) being occupied.

infinisil commented 3 months ago

We previously decided to deprecate the directory mode of nixfmt and instead encourage treefmt, for which I recently implemented parallel formatting.

So I don't think there's anything to do here, let's keep the complexity of this codebase at a minimum :)

piegamesde commented 3 months ago

I think that there is quite a bit of code performance which Nixfmt currently leaves on the table. Someone with knowledge of profiling Haskell applications would likely find quite a few low-hanging fruits.

Off the top of my head I can already think of https://github.com/NixOS/nixfmt/blob/master/src/Nixfmt/Predoc.hs#L178-L192 which has overall quadratic complexity, but by delaying computations into a single pass at the end this could be reduced to linear. (I didn't do it when I wrote the code because it requires a minor data types refactoring and I wanted to get stuff to work first.) Of course I can't tell how much this actually affects overall performance, but there are probably quite a few like this which add up.