Feature Request: Conditional skipping test based on setup scripts

Motivation

In large-scale projects or projects heavily integrated with third-party services, it is often necessary to skip certain tests when specific conditions are not met. For instance, if a required database is not available, running integration tests dependent on this database should be skipped to save time and avoid irrelevant test failures.

Existing solutions

Rust issue #68007: A long-standing issue suggests runtime test skipping but hasn't progressed. Need fundamental change on libtest.
Community crate test-with: This uses a proc macro to add #[ignore] at compile time. While useful, it lacks the flexibility.

Proposal

Enhance nextest to support conditional test skipping based on the output of a setup script. The configuration may look like:

[[profile.default.scripts]]
filter = "test(~integration)"
run-if = 'check-db'
# alternatively, we can implement an option `skip-if-fail`
# setup = 'check-db'
# skip-if-fail = true

[script.check-db]
command = 'cargo run -p check-db'

When the script outputs 'false', the tests in the specified group (filtered by "test(~integrate)") should be marked as SKIP.

Finished `test` profile [unoptimized + debuginfo] target(s) in 0.13s
Starting 5 tests across 1 binaries (run ID: 7d7253f2-e050-46cd-98ef-0815cff44a4c, nextest profile: default)
    PASS [   0.002s] ut::foo
    PASS [   0.003s] ut::bar
    PASS [   0.003s] ut::baz
    SKIP [   0.003s] integration::foo
    SKIP [   0.003s] integration::bar
------------
    Summary [   0.022s] 5 tests run: 3 passed, 2 skipped

On CI, we can run cargo nextest run --run-ignored all to force them to run.

nextest-rs / nextest