False negative for tests with datasets

[V13Axel]

If a test has ->with(), it technically changes into multiple tests. As such, passes are detected as failures:

I'm guessing it's because the test ID(s) rely on the string passed to it() or test(), and expect that 1 it() or test() === 1 test. When using datasets, that's not the case. Hrm.

Will need to chew on how to actually handle that in the context of this adapter.

[V13Axel]

After looking into this a bit further, it appears the term used for this elsewhere is "parameterized" tests, and neotest does support them. I'll be looking into the way neotest-jest handles them for inspiration, and hope to have that solved sometime in the near future.

[anhoder]

+1

[calebdw]

I want to point out that the same problem also happens when using `->repeat()

[V13Axel]

Thanks @calebdw - I've been experimenting with this a bit already, so knowing that will help me make things a bit more generalized

[sixlive]

Any headway on this?

[V13Axel]

@sixlive nothing concrete yet, though I continue to experiment with it. I'll definitely post an update here just as soon as there is something to share!

[V13Axel]

... After quite a few hours and a number of attempts at test discovery using tree-sitter, I have discovered that there is a --list-tests flag that can be passed to both pest and phpunit that will generate a list of tests - and it even lists parameterized tests separately!

I'm not 100% sure what that might end up looking like here either, but I'll be evaluating it as a path forward.

[sixlive]

Awesome, thanks for the update. I was hacking around on it last weekend and was also struggling trying to get things working via tresitter. I think the --list-tests is a promising way forward. Might try hacking on it again this weekend if I end up with some free time. I'll post back if I end up with anything interesting.

[victor-falcon]

I'm facing this issue with a normal it test… 🤔