search

nix-community / nix-eval-jobs

Parallel nix evaluator with a streamable json output [maintainers @Mic92, @adisbladis]
GNU General Public License v3.0
145 stars 27 forks source link

Output evaluation statistics #251

Open danielfullmer opened 11 months ago

danielfullmer commented 11 months ago

It would be nice if nix-eval-jobs could also output evaluation statistics in the JSON output for each derivation, similar to what we get from nix-build when setting NIX_SHOW_STATS=1:

$ NIX_SHOW_STATS=1 nix-build . -A hello
{
  "cpuTime": 0.07112400233745575,
  "envs": {
    "bytes": 149776,
    "elements": 8144,
    "number": 5289
  },
  "gc": {
    "heapSize": 402915328,
    "totalBytes": 7441872
  },
  "list": {
    "bytes": 21416,
    "concats": 55,
    "elements": 2677
  },
  "nrAvoided": 7500,
  "nrFunctionCalls": 4661,
  "nrLookups": 1970,
  "nrOpUpdateValuesCopied": 283430,
  "nrOpUpdates": 448,
  "nrPrimOpCalls": 3363,
  "nrThunks": 43562,
  "sets": {
    "bytes": 5303360,
    "elements": 329564,
    "number": 1896
  },
  "sizes": {
    "Attr": 16,
    "Bindings": 16,
    "Env": 16,
    "Value": 24
  },
  "symbols": {
    "bytes": 203048,
    "number": 21009
  },
  "values": {
    "bytes": 1350384,
    "number": 56266
  }
}
this path will be fetched (0.05 MiB download, 0.22 MiB unpacked):
  /nix/store/g2m8kfw7kpgpph05v2fxcx4d5an09hl3-hello-2.12.1
copying path '/nix/store/g2m8kfw7kpgpph05v2fxcx4d5an09hl3-hello-2.12.1' from 'https://cache.nixos.org'...
/nix/store/g2m8kfw7kpgpph05v2fxcx4d5an09hl3-hello-2.12.1
Mic92 commented 11 months ago

The question is though when to print this. nix-eval-jobs has several background workers each with their own statistics and workers are restarted if they consume too much memory. This makes just using evaluation statistics provided by nix itself hard to use.