search

nix-community / crate2nix

rebuild only changed crates in CI with crate2nix and nix
https://nix-community.github.io/crate2nix/
Apache License 2.0
352 stars 83 forks source link

Allow test crate to fail but still "build" #191

Closed Fuuzetsu closed 3 years ago

Fuuzetsu commented 3 years ago

We are running tests in multiple crates, creating the test report with something like cargo2junit and showing it in the CI. But if the crate just fails, we can't see the (failing) output and handle the failure reporting ourselves.

Would be cool to unconditionally produce test output even if it failed.

Behind some flag of course.

kolloch commented 3 years ago

Thank you for the suggestion!

One common problem with this approach is:flaky tests are not rerun. If your tests are fully determistic, this is not a problem, of course.

kolloch commented 3 years ago

Depending on your setup, the errors are often in the build log. Would be interesting to hear what others think.

Fuuzetsu commented 3 years ago

Right, obviously this sucks for flaky tests, hence the opt-in. We don't have flaky tests and if re-running the test, it should just re-use the result.

I will submit something today.