mitsuhiko / insta

A snapshot testing library for rust
https://insta.rs
Apache License 2.0
2.26k stars 102 forks source link

Add support for #[test_case] crate #684

Open NicolasWent opened 2 weeks ago

NicolasWent commented 2 weeks ago

Hello,

When I use test_case crate with insta, I get wrong test name results and the tests are not passing when I don't modify the code and re-run them

Minimal reproducing example:

Cargo.toml:

[package]
name = "insta-suggestion"
version = "0.1.0"
edition = "2021"

[dependencies]
test-case = "3.3"
insta = { version = "1.41", features = ["yaml"] }

lib.rs:

use insta::assert_yaml_snapshot;
use test_case::test_case;

pub fn to_upper_case(text: &str) -> String {
    text.to_uppercase()
}

#[test_case("Hello World" ; "test_hello_world")]
#[test_case("Insta is fun" ; "test_insta_is_fun")]
fn test_function(text: &str) {
    let res = to_upper_case(text);
    assert_yaml_snapshot!(res);
}

What happens

Running cargo test will create two files: insta_suggestion__function-2.snap.new and insta_suggestion__function.snap.new

If I re-run cargo test, it will fail everytime

Expected output

The two files created should have the right name with "test_hello_world" and "test_insta_is_fun" instead of "test_function"

Aditionaly, if we re-run cargo test without changing anything, the tests shouldn't fail

Why this would be a good features

Pros

insta seems to be a crate designed to write tests faster, test_case also. It would be amazing to combine both and be able to write lots of tests very fast

Cons

Some features, specially with the @"" keyword or the vscode extension will be hard to work with

CommanderStorm commented 2 days ago

cannot reproduce using the example above. Did you review/accept the snapshots?

image

CommanderStorm commented 2 days ago

from insta's POV, the naming of the testcase correct. The functionon gets expanded to this, with test_case specific logic in a different scope. The function here is called multiple times => this is why you are seeing function-N as the snapshot name.

image

CommanderStorm commented 2 days ago

Discussions about interop between the crates is here:

=> using insta::with_settings seems like the best way that the community has come up with to tag this if using the function-N naming for testcases does not work for you.

NicolasWent commented 1 day ago

Thanks a lot

Yes I forgot to do cargo insta review sorry I am new to insta :'(

It works after cargo insta review :)

Could you show me an example of using it to rename my tests correctly?

NicolasWent commented 1 day ago

After using test case and insta in a bigger project I am getting the issue that the tests are getting "shuffled" like running multiple times cargo test ask me to review snapshots that were already reviewed

CommanderStorm commented 1 day ago

Could you show me an example of using it to rename my tests correctly

An example how suffixes can be set can be found here:

The same way also holds for the other settings.

CommanderStorm commented 1 day ago

tests are getting "shuffled" like running multiple times cargo test ask me to review snapshots that were already reviewed

What commands are you running and how are your test-cases structured? If for example the call-order is non-deterministic, using call numbers as a suffix won't work reliably.