log checks do not work on darwin

[SuperSandro2000]

I noticed while using nixpkgs-review-checks that hammerings log features do not work on darwin:

$ nix-build -A wrangler
/nix/store/y87ga2kkrdb8xq9fpc6zbmk25h4k8bp4-wrangler-1.13.0
$ nixpkgs-hammer wrangler
When evaluating attribute ‘wrangler’:
warning: EvalError
Cannot evaluate attribute ‘wrangler’ in ‘/nix/store/06qbd3qw844pq525489ki17xnyb1jlpf-nixpkgs’.
notice: no-build-output
‘wrangler’ has not yet been built. Checks that rely on the presence of build logs are skipped.
$ nix log /nix/store/y87ga2kkrdb8xq9fpc6zbmk25h4k8bp4-wrangler-1.13.0
...

I am not sure what I am doing wrong here. If this is not a bug we should add a few words to get it working.

[rmcgibbo]

Yeah, that looks off, for sure. Especially the 'EvalError'. Do you have any examples where it doesn't show the EvalError and the log checks still fail by chance?

[jtojnar]

The issue is that it does not evaluate. Could you check if running with --do-not-catch-errors from https://github.com/jtojnar/nixpkgs-hammering/pull/88 gives any useful info?

[SuperSandro2000]

This happens for wranger and git. I think this is a bug somewhere because git builds on darwin and has no qt in its dependencies.

command nix run -f https://github.com/jtojnar/nixpkgs-hammering/archive/61a2da5490edceda98626e40f656f5b5c36c4cd9.tar.gz -c nixpkgs-hammer --do-not-catch-errors wrangler
error: --- ThrownError ----------------------------------------------------------------------------------------------------------------------------------------------------- nix-instantiatePackage ‘qt-4.8.7’ in /Users/user/nixpkgs/pkgs/development/libraries/qt-4.x/4.8/default.nix:233 is not supported on ‘x86_64-darwin’, refusing to evaluate.

a) To temporarily allow packages that are unsupported for this system, you can use an environment variable
   for a single invocation of the nix tools.

     $ export NIXPKGS_ALLOW_UNSUPPORTED_SYSTEM=1

b) For `nixos-rebuild` you can set
  { nixpkgs.config.allowUnsupportedSystem = true; }
in configuration.nix to override this.

c) For `nix-env`, `nix-build`, `nix-shell` or any other Nix command you can add
  { allowUnsupportedSystem = true; }
to ~/.config/nixpkgs/config.nix.

(use '--show-trace' to show detailed location information)
Traceback (most recent call last):
  File "/nix/store/2yy82yl5wkzm7kxqlir5lk58dx6kvd4a-nixpkgs-hammer/bin/.nixpkgs-hammer-wrapped", line 407, in <module>
    main(args)
  File "/nix/store/2yy82yl5wkzm7kxqlir5lk58dx6kvd4a-nixpkgs-hammer/bin/.nixpkgs-hammer-wrapped", line 328, in main
    overlay_data = nix_eval_json(all_messages_nix, args.show_trace)
  File "/nix/store/2yy82yl5wkzm7kxqlir5lk58dx6kvd4a-nixpkgs-hammer/bin/.nixpkgs-hammer-wrapped", line 136, in nix_eval_json
    json_text = subprocess.check_output(args, text=True, input=expr)
  File "/nix/store/xf3244kbn36z04yw98mpgvvsgdcbz438-python3-3.8.8/lib/python3.8/subprocess.py", line 415, in check_output
    return run(*popenargs, stdout=PIPE, timeout=timeout, check=True,
  File "/nix/store/xf3244kbn36z04yw98mpgvvsgdcbz438-python3-3.8.8/lib/python3.8/subprocess.py", line 516, in run
    raise CalledProcessError(retcode, process.args,
subprocess.CalledProcessError: Command '['nix-instantiate', '--strict', '--json', '--eval', '-']' returned non-zero exit status 1.

[jtojnar]

Maybe from https://github.com/jtojnar/nixpkgs-hammering/blob/b59787fea072a9a09c5bc22b9fec0ac1272d81f0/overlays/build-tools-in-build-inputs.nix#L72

[jtojnar]

Yeah. And also automoc4, breakpointHook, gogUnpackHook and sudo according to nix-instantiate --argstr system x86_64-darwin -A "antlr" … --show-trace.

Perhaps we should use allowUnsupportedSystem = true. Or just filter these like we do for darwin.cctools Linux? Do not really like either very much.

[SuperSandro2000]

Or just filter these like we do for darwin.cctools Linux?

That wouldn't be future proof.

Also found another issue on linux:

$ command nix run -f https://github.com/jtojnar/nixpkgs-hammering/archive/61a2da5490edceda98626e40f656f5b5c36c4cd9.tar.gz -c nixpkgs-hammer --do-not-catch-errors git
error: --- ThrownError ----------------------------------------------------------------------------------------------------------------------------------------------------- nix-instantiateurllib3-1.26.3 not supported for interpreter python2.7
(use '--show-trace' to show detailed location information)
Traceback (most recent call last):
  File "/nix/store/pj7qk83bqgy33b8fxx077rn9h6jdrmp7-nixpkgs-hammer/bin/.nixpkgs-hammer-wrapped", line 407, in <module>
    main(args)
  File "/nix/store/pj7qk83bqgy33b8fxx077rn9h6jdrmp7-nixpkgs-hammer/bin/.nixpkgs-hammer-wrapped", line 328, in main
    overlay_data = nix_eval_json(all_messages_nix, args.show_trace)
  File "/nix/store/pj7qk83bqgy33b8fxx077rn9h6jdrmp7-nixpkgs-hammer/bin/.nixpkgs-hammer-wrapped", line 136, in nix_eval_json
    json_text = subprocess.check_output(args, text=True, input=expr)
  File "/nix/store/yl69v76azrz4daiqksrhb8nnmdiqdjg9-python3-3.8.8/lib/python3.8/subprocess.py", line 415, in check_output
    return run(*popenargs, stdout=PIPE, timeout=timeout, check=True,
  File "/nix/store/yl69v76azrz4daiqksrhb8nnmdiqdjg9-python3-3.8.8/lib/python3.8/subprocess.py", line 516, in run
    raise CalledProcessError(retcode, process.args,
subprocess.CalledProcessError: Command '['nix-instantiate', '--strict', '--json', '--eval', '-']' returned non-zero exit status 1.

Edit:

Perhaps we should use allowUnsupportedSystem = true.

Probably because the linter should also work on not supported packages.