`ExceptionGroup` ergonomics in `pytest.param(..., marks=xfail(raises=...))`

[Zac-HD]

Suppose you have a parameterized test, some params of which are expected to raise (e.g.) IndexError:

return pytest.param(
    ...,
    marks=pytest.mark.xfail(raises=IndexError, strict=True),
)

If your test is async though, you'll actually raise an ExceptionGroup(..., [IndexError]) (maybe with even more nesting), so the tests will still fail. Can we make this more ergonomic? What would a parametrize-aware RaisesGroup-like thing look like?

See also https://github.com/pytest-dev/pytest/issues/11538 and https://github.com/pytest-dev/pytest/pull/11671.

[jakkdl]

I might be seeing nails everywhere, but I would do

return pytest.param(
    ...,
    marks=pytest.mark.xfail(raises=RaisesGroup(IndexError), strict=True),
)

and afaict the only code doing the matching is https://github.com/pytest-dev/pytest/blob/d9a2fd04635c57100f25e93d7d98317d824b2e69/src/_pytest/skipping.py#L280-L281 where it would be straightforward to add

if isinstance(raises, RaisesGroup) and not raises.matches(call.excinfo.value):
  rep.outcome = "failed"
elif raises is not None [...]
  ...

This of course depends on #11671 (or a version that fully implements trio.RaisesGroup functionality).

[jakkdl]

Actually, I don't see why this is about pytest.param at all. If pytest.mark.xfail(raises=...) got support for RaisesGroup (or equivalent) it should Just Work :tm:

@pytest.mark.xfail(raises=RaisesGroup(ValueError))
def test_foo():
    raise ExceptionGroup("foo", [ValueError])

11671 has fallen behind the trio implementation, and is stalled by people wanting to have opinions on the interface design, otherwise I'd resolve this issue in it right away.