phoronix-test-suite: the test did not produce a result

[Zahrun]

Describe the bug

In phoronix-test-suite at least the test unigine-heaven fails, probably others too.

Steps To Reproduce

Steps to reproduce the behavior:

1. phoronix-test-suite benchmark unigine-heaven
2. Select resolution (4 for 1920 x 1080)
3. Select fullscreen or windowed
4. Say no to save results to gain time

Expected behavior

test runs

Logs

    Would you like to save these test results (Y/n): n

Unigine Heaven 4.0:
    pts/unigine-heaven-1.6.5 [Resolution: 1920 x 1080 - Mode: Windowed - Renderer: OpenGL]
    Test 1 of 1
    Estimated Trial Run Count:    3                      
    Estimated Time To Completion: 16 Minutes [22:52 UTC] 
        Started Run 1 @ 22:37:16
        The test run did not produce a result.
        Started Run 2 @ 22:37:21
        The test run did not produce a result.
        Started Run 3 @ 22:37:25
        The test run did not produce a result.
        E: unigine-heaven: line 20: ./bin/heaven_x64: No such file or directory

Notify maintainers

@davidak

Metadata

Please run nix-shell -p nix-info --run "nix-info -m" and paste the result.

[user@system:~]$ nix-shell -p nix-info --run "nix-info -m"
 - system: `"x86_64-linux"`
 - host os: `Linux 5.15.102, NixOS, 22.11 (Raccoon), 22.11.3196.cd34d6ed7ba`
 - multi-user?: `yes`
 - sandbox: `yes`
 - version: `nix-env (Nix) 2.11.1`
 - channels(aroun): `""`
 - channels(root): `"nixos-22.11"`
 - nixpkgs: `/nix/var/nix/profiles/per-user/root/channels/nixos`

[davidak]

In phoronix-test-suite at least the test unigine-heaven fails, probably others too.

Yes, many tests fail. Often because they need additional dependencies. I don't think it's a good idea to add many dependencies, so every test works.

Instead, i think it would make sense to document how to run tests that need specific dependencies.

What do you think?

Would you rather download many dependencies, even if you don't need them or have to look up documentation to get specific tests working?

[bjornfor]

Would you rather download many dependencies, even if you don't need them or have to look up documentation to get specific tests working?

It depends. (How big would a "full" closure be?) However, can't we "get around" this issue by adding all the deps we need to phoronix-test-suite, and then having an additional phoronix-test-suite-minimal for those who consider that bloat? (I'm a big fan of it-works-out-of-the-box.)

[davidak]

I found out again that phoronix-test-suite does it's own package management by using the system package manager. So for the most elegant solution, we would need to add Nix support. Best would be to use nix shell to not actually install the dependencies.

https://github.com/phoronix-test-suite/phoronix-test-suite/blob/master/documentation/phoronix-test-suite.md#external-dependencies

https://github.com/phoronix-test-suite/phoronix-test-suite/blob/master/documentation/phoronix-test-suite.md#debug-dependency-handler https://github.com/phoronix-test-suite/phoronix-test-suite/blob/master/documentation/phoronix-test-suite.md#dump-unhandled-dependencies

https://github.com/phoronix-test-suite/phoronix-test-suite/tree/master/pts-core/external-test-dependencies/scripts https://github.com/phoronix-test-suite/phoronix-test-suite/tree/master/pts-core/external-test-dependencies/xml

Does anyone want to contribute that upstream?

[bjornfor]

@davidak: Thanks for the info! But I'm afraid nix-shell won't work here, as it creates sub-shells, whereas upstream probably assumes "installing the packages" makes them available also for parent processes.

[bjornfor]

(And I wouldn't want upstream to call nix-env automatically -- side effects -- eek!)

[Zahrun]

In phoronix-test-suite at least the test unigine-heaven fails, probably others too.

Yes, many tests fail. Often because they need additional dependencies. I don't think it's a good idea to add many dependencies, so every test works.

Instead, i think it would make sense to document how to run tests that need specific dependencies.

What do you think?

Would you rather download many dependencies, even if you don't need them or have to look up documentation to get specific tests working?

I think documentation is a good option.

Another one would be to make the package modular, as in it can take arguments depending on which benchmark we want to have like phoronix-test-suite { unigine-heaven }. Or it could be a plugin-like type of package, such as we have for other software:

(unstable.wrapOBS {
      plugins = [
        obs-studio-plugins.obs-backgroundremoval
        #obs-ndi
      ];
})

or

(rstudioWrapper.override { packages = with rPackages;
    [
      rsm # response surface methodology
      Rcmdr # a gui for R
      RcmdrPlugin_DoE # Design of Experiements plugin
      sem rmarkdown rgl multcomp markdown lmtest leaps aplpack # Rcmdr optional dependencies
    ]; })

What do you think?