search

NixOS / nixpkgs

Nix Packages collection & NixOS
MIT License
18.05k stars 14.09k forks source link

Python: run tests in separate derivation #26400

Open FRidh opened 7 years ago

FRidh commented 7 years ago

Issue description

Currently we run tests directly after a package has been build. Unfortunately, we need to disable tests regularly to avoid circular dependencies caused by test dependencies.

Running tests in a separate derivation would:

I propose we then add a tested attribute to Python packages that corresponds to the second derivation. This is typically the attribute to use for CI as well.

copumpkin commented 7 years ago

I've often wanted this for other types of derivations, too, but I'd definitely support this.

anderspapitto commented 7 years ago

One potential snag - there are probably packages where some tests rely on access to temporary build files. In those cases, you'd either have to disable any such tests, or alternatively the tested attribute could redo the whole build.

peterhoeg commented 7 years ago

Earlier conversation #9281.

deliciouslytyped commented 5 years ago

It's also very annoying during packaging / nix package development when a test fails after an hour long build and you have to go through the whole process again just to work on fixing the tests. (Ok, you can use nix-shell)

stale[bot] commented 4 years ago

Thank you for your contributions.

This has been automatically marked as stale because it has had no activity for 180 days.

If this is still important to you, we ask that you leave a comment below. Your comment can be as simple as "still important to me". This lets people see that at least one person still cares about this. Someone will have to do this at most twice a year if there is no other activity.

Here are suggestions that might help resolve this more quickly:

  1. Search for maintainers and people that previously touched the related code and @ mention them in a comment.
  2. Ask on the NixOS Discourse.
  3. Ask on the #nixos channel on irc.freenode.net.
doronbehar commented 3 months ago

I don't think we should (by default) add such a tested attribute, or put the tests in a passthru.tests derivation. In fact, I think that building the tests in the same derivation has the advantage that it forces us to observe the failure, which in turn can warn us about failures also in dependent packages. The same reasoning also pushes me to advocate in transitioning from adding a passthru.tests.version to versionCheckHook. See also: