Thanks for the library, I use it extensively in one of my projects with great success.
Recently, I ran into an unfortunate edge-case in the default directory selection logic. Specifically, I wrote a parameterized test whose parameter values were only differentiated by special (non-path-safe) characters. The regex which sanitizes generated directory names removed the special characters, meaning that all variants of the parameterized test chose the same directory name. Each test run then overwrote the last one's snapshot.
I could imagine scenarios where this would silently give extremely unexpected results. Even if it isn't silent (hopefully the test fails due to differing snapshots!), it might take significant time to debug.
I think, ideally, the library would warn when multiple tests alias to the same directory. However, I'm not sure this is straightforward, since pytest tears down the environment between runs and I am not personally aware of a way to retrieve the test list in advance (although I'd love to find out otherwise). The alternative solution I see would be to modify the test naming logic to stringify special characters as their Unicode code point numbers, names, or similar. Or perhaps just erroring in that case and requiring a manual override.
I'm not sure there's a non-breaking fix here (otherwise I'd have opened a PR). But I figured I'd open the issue and see if others have thoughts.
Hi!
Thanks for the library, I use it extensively in one of my projects with great success.
Recently, I ran into an unfortunate edge-case in the default directory selection logic. Specifically, I wrote a parameterized test whose parameter values were only differentiated by special (non-path-safe) characters. The regex which sanitizes generated directory names removed the special characters, meaning that all variants of the parameterized test chose the same directory name. Each test run then overwrote the last one's snapshot.
I could imagine scenarios where this would silently give extremely unexpected results. Even if it isn't silent (hopefully the test fails due to differing snapshots!), it might take significant time to debug.
I think, ideally, the library would warn when multiple tests alias to the same directory. However, I'm not sure this is straightforward, since pytest tears down the environment between runs and I am not personally aware of a way to retrieve the test list in advance (although I'd love to find out otherwise). The alternative solution I see would be to modify the test naming logic to stringify special characters as their Unicode code point numbers, names, or similar. Or perhaps just erroring in that case and requiring a manual override.
I'm not sure there's a non-breaking fix here (otherwise I'd have opened a PR). But I figured I'd open the issue and see if others have thoughts.