search

Mic92 / nixpkgs-review

Review pull-requests on https://github.com/NixOS/nixpkgs
MIT License
381 stars 63 forks source link

`passthru.tests` should be run automatically #297

Closed erikarvstedt closed 1 year ago

erikarvstedt commented 1 year ago

Package update PR https://github.com/NixOS/nixpkgs/pull/206835 broke the corresponding module (later fixed here).

This could have been prevented if the standard review command (nixpkgs-review pr 206835) would automatically run the package-specific NixOS tests defined in passthru.tests.

Instead, reviewers need to manually run the test with an extra command:

nixpkgs-review pr 206835
nixpkgs-review pr -p nixosTests.paperless 206835

This is error-prone, not least because the NixOS test name can differ from the package name (like in this case).

cc @NobbZ, @SuperSandro2000

NobbZ commented 1 year ago

There should be a way to opt out from running the tests then, as they might involve virtualisation, which might not be available for everyone.

Any generated output, and perhaps the PR template should be adjusted then to also tell whether tests or not have been (successfully) ran.

Mic92 commented 1 year ago

Duplicate of https://github.com/Mic92/nixpkgs-review/issues/77