Open JohnstonJ opened 3 weeks ago
Here's an updated workaround I came up with. It solves the problem of somebody forgetting to add a #[case]
attribute by moving all the test cases into a single HashMap
, and then adding a test case that asserts that every map entry has a corresponding test in the test binary.
The test_case_map
macro has an example that shows how to use it.
I don't particularly love this solution, but I haven't been able to think of a better way. It makes some assumptions about the output of the test binary when using --list
, it has to invoke the test binary in the first place, it makes some assumptions about the naming convention of the rstest-generated code, and the function_name!
macro that was used makes some assumptions about how Rust returns this string.
Potentially rstest could generate more code that reduces/eliminates the need for some of these assumptions, but then again it would also be adding new public APIs for what amounts to a rustfmt workaround, so not sure if it's worth it.
For example, rstest could provide an automatic "test case name" fixture so I don't have to call function_name!
. It could also provide a fixture for the name of the test itself. (Something that seems to be a common request but hasn't yet been implemented by Rust's testing infrastructure - no good solutions to get at runtime. But rstest, being a macro, can easily discover the test name during parsing.) It could also emit a static variable that contains the list of all test cases for the test (names of generated test functions).
For function name you can take a look to #177 as example. I would like to implement a #[context]
attribute but I don't know if I can find some time to do it
Just to note that everybody can write and publish crate with fixture implementations these crate should not be included in rstest
but just use it.
Yeah.... using the thread name feels like an even worse undocumented hack though. I tried out that approach as well, but I wasn't comfortable with it because I wasn't sure how reliable it would be. I noticed there has been some instability in the thread name in the past, like if the number of test threads is changed to be single-threaded.
It seems a lot of people are asking for a stable API to get the test name, but the requests are several years old, so who knows if/when it will happen. In the meantime, it seems like a useful thing that rstest could provide in a much more stable/supported way.
I would like to implement a #[context] attribute
If by this, you mean allowing the injection of a parameter to the test that has context information about the test (e.g. test name, other things) - i.e. similar to Go's testing.T type, which provides a name() function, then I think it's a great idea!
Just to note that everybody can write and publish crate with fixture implementations these crate should not be included in rstest but just use it.
For the function name macro I referenced, I think it is impossible to put it in a fixture. It would just return the name of the fixture function instead of the test case name. I looked at the code generated by rstest, and it seems that each test case results in a separate generated function which calls the original user-provided function. The call to this function name macro has to happen inside the generated test case function in order to come up with the test case name.
TL;DR: I want my test case inputs into rstest to be auto-reformatted with rustfmt, and this is proving surprisingly hard to figure out a way to do this without introducing other downsides like the possibility of forgetting to run a test case.
I have some more complicated test case inputs, and I'm struggling to figure out how to use
rstest
so that they are automatically formatted by rustfmt. Is it even possible in a reasonable way? If not, I wonder if there may be some solutions possible to make the situation better.Currently, it seems that anything inside of
#[case::panic(42, "Foo")]
is not reformatted by rustfmt, and this can get quite messy if we are passing in a lot of parameters and/or complex structs.The best workaround I've come up with is to put each test case into its own separate variable, and reference the variables. Here is an example:
Click to view example
```rust #[derive(Debug)] struct InfoCheckSimilarTestCase<'a> { expected: Info, comparison: Info, err: Option<&'a str>, } static INFO_CHECK_SIMILAR_MATCHES: LazyLockWhile this does restore the ability to format the complex test cases with rustfmt because at this point they are simple static variables, this workaround has several downsides:
#[case(...)]
, then the test case is not run!Ideally, there would be a way to have
#[rstest]
consume a list of test cases from someplace else? But I haven't been able to figure out how to do that. The matrix / list input testing functionality still seems to demand a hard-coded list, which still leaves opportunity for somebody to forget to add new test case variables to the rstest matrix.Key things I'm looking for:
Past experiences in other languages:
case
syntax, Python's black reformatter has no problem with reformatting code inside a decorator. So I put the test case structures directly inside of the decorator, similar to the rstest examples in the documentation.In comparison, this is proving surprisingly hard for me to figure out how best to do this in Rust...