possible to allow exceptions during test setup?

[svenevs]

Hello, here's an odd scenario. I was actually looking to implement it myself, but saw you had this nice and packaged all ready to go so I'm game to contribute if you have suggestions / think it is doable.

The way my testing framework works is kind of frustrating, but necessary. I'm testing a sphinx extension I wrote, and the general pattern here is

def test_something(self):
    # ... my extension has already run by this point ...

we had to do some pretty gnarly metaclassing and fixtures in order to cooperate with Sphinx, but basically this will work

@pytest.mark.xfail()
@my_decorator_that_forces_an_exception
def test_something(self):
    pass

But using your @pytest.mark.raises(exception=sphinx.errors.ConfigError) instead, it is marked as a hard failure. I think it comes from yield being first:

https://github.com/Authentise/pytest-raises/blob/f07c75132898eb92fd4936ca3929af103f3dec2e/pytest_raises/pytest_raises.py#L17

TBH I'm still kind of a pytest n00b. I tried something probably overly-naive like

try:
    outcome = yield
except Exception as e:
    outcome = e

# ...

but I don't really understand how this all works.

[svenevs]

WOOT I figured it out! So for me, pytest_runtest_call is "too late", since the exception occurs during the sphinx marker. I don't think you can have it both ways, but I think in general pytest_runtest_call would be the average use case. I will include my changes here without a PR because I don't know why I had to change some of them...

1. I had to use pytest_runtest_setup, not pytest_runtest_call. Note that exceptions raised in the test body (if applicable) will now need to be used with traditional with pytest.raises(...).
2. Probably related to changing (1), I would get attribute errors (unable to set) for the outcome.excinfo = .... Though probably not reliable, __dict__ crawling revealed I can set _excinfo.
3. This is probably the only one you want to include here. In the else clause at the bottom, doing raise raised_exception and only excepting on an ExpectedException is problematic. This will crash pytest completely, rather than letting the test "gracefully fail". My solution was to elif raised_exception with a slightly different message ;)

Here is the diff I am using and just added to my local conftest.py

diff --git a/pytest_raises/pytest_raises.py b/pytest_raises/pytest_raises.py
index 3987ea1..18e73f7 100644
--- a/pytest_raises/pytest_raises.py
+++ b/pytest_raises/pytest_raises.py
@@ -13,7 +13,7 @@ class ExpectedMessage(Exception):

 @pytest.hookimpl(hookwrapper=True)
-def pytest_runtest_call(item):
+def pytest_runtest_setup(item):
     outcome = yield
     raises_marker = item.get_marker('raises')
     if raises_marker:
@@ -33,15 +33,26 @@ def pytest_runtest_call(item):
                 except(ExpectedMessage):
                     excinfo = sys.exc_info()
                     if traceback:
-                        outcome.excinfo = excinfo[:2] + (traceback, )
+                        outcome._excinfo = excinfo[:2] + (traceback, )
                     else:
-                        outcome.excinfo = excinfo
+                        outcome._excinfo = excinfo
+        elif raised_exception:
+            try:
+                raise ExpectedException('Expected exception {}, but got {} with message: {}'.format(
+                    exception, type(raised_exception), raised_exception
+                ))
+            except ExpectedException:
+                excinfo = sys.exc_info()
+                if traceback:
+                    outcome._excinfo = excinfo[:2] + (traceback, )
+                else:
+                    outcome._excinfo = excinfo
         else:
             try:
-                raise raised_exception or ExpectedException('Expected exception {}, but it did not raise'.format(exception))
-            except(ExpectedException):
+                raise ExpectedException('Expected exception {}, but it did not raise'.format(exception))
+            except ExpectedException:
                 excinfo = sys.exc_info()
                 if traceback:
-                    outcome.excinfo = excinfo[:2] + (traceback, )
+                    outcome._excinfo = excinfo[:2] + (traceback, )
                 else:
-                    outcome.excinfo = excinfo
+                    outcome._excinfo = excinfo

Hopefully this all will be useful to somebody some day if they ever have such specialized needs! @Authentise thanks for making pytest_raises available, with an agreeable license, so that I can include this hacked together stuff directly in my library (with proper attribution of course xD).