la10736 / rstest

Fixture-based test framework for Rust
Apache License 2.0
1.21k stars 43 forks source link

Reusing test data with different expected output across tests? #275

Open Seanmatthews opened 2 months ago

Seanmatthews commented 2 months ago

I can't find a way to do this with either rstest or rstest_reuse, and it'd be very useful for my case. I have a bunch of defined instantiations of a struct against which I'd like to test different operations. But while each of these test functions (one for each operation) take the same list of struct data, the expected output of the operation is different. Is there a way to combine the test data with different expected results for each test?

The closest I got was with something like this:

#[template]
#[rstest]
#[case(2, 2)]
#[case(2, 3)]
fn two_simple_cases(#[case] a: u32, #[case] b: u32) {}

#[apply(two_simple_cases)]
fn it_works(a: u32, b: u32, #[values(true, false)] expected: bool) {
    assert_eq!(a == b, expected);
}

However, this performs every combination of the input and the expected value: (2, 2, true), (2, 2, false), (2, 3, true), (2, 3, false). How can I do something like the above that will result in the cases: (2, 2, true), (2, 3, false)?

Seanmatthews commented 2 months ago

Expanding on this-- it really comes down to the #[values()] feature creating every combination of inputs. Which, that provides great generation ability, but precludes the ability to associate test inputs & expected values in a 1:1 fashion.

How would you feel about a #[values_assoc()] feature, which requires the same number of cases from other sources, and associates arguments in a 1:1 fashion? Example:

fn test(#[values_assoc(a, b)] val: &str, #[values_assoc(1, 2)] num: i32) {}

would generate test cases

test(a, 1)
test(b, 2)

Is this a feature for which you'd welcome a PR?

la10736 commented 2 months ago

Sorry, but I cannot understand what's this is not equivalent to

#[rstest]
fn test(#[values(("a", 1), ("b", 2))] vals: (&str, i32)) {}

or

#[rstest]
#[case("a", 1)]
#[case("b", 2)]
fn test(val: &str, num: i32) {}

I got your point that's how this feature can work with rstest_reuse but, maybe, is better to find a syntax to use to enrich the apply macro and attach some other parameters to templates.

la10736 commented 2 months ago

I guess that this ticket is quite the same of #215