Return exit code from ipytest.run

[chmp]

Makes it easer to check for errors when running the tests. See #62

[itepifanio]

@chmp are you working on some option to stop the notebook if the tests break? This issue is not much clear to me.

I have the same problem that @ralfschulze mentioned on #62: my CI/CD ignores if the test break. It would be very cool to have some option to check the exit_code automatically.

[chmp]

At the moment, I only planned to return the exit code from ipytest.run, such that the following works:

assert ipytest.run() == 0

Are you using the function or the cell magic to execute your tests?

[itepifanio]

I'm using cell magic. Currently, I'm using the ipytest.exit_code in the next cell of the test to make sure if has passed/fail.

%%ipytest

def test_it_shall_not_pass():
    assert False == True

# next cell 
assert ipytest.exit_code == 0

I tried to use assert ipytest.run() == 0 on the same cell, but it always returns None. For example, the next test returns the following trace:

%%ipytest

def test_it_shall_pass():
    assert True == True

assert ipytest.run() == 0

---------------------------------------------------------------------------
AssertionError                            Traceback (most recent call last)
<ipython-input-45-a814d4048647> in <module>
      2     assert True == True
      3 
----> 4 assert ipytest.run() == 0

AssertionError: assert None == 0
 +  where None = <function run at 0x7f8a3edf9550>()
 +    where <function run at 0x7f8a3edf9550> = <module 'ipytest' from '/home/italo/.cache/pypoetry/virtualenvs/ipyannotator-SET3dKKr-py3.8/lib/python3.8/site-packages/ipytest/__init__.py'>.run

[chmp]

That's expected, as the exit code is only set after the cell has been executed. So the assert sees the exit code of the previous invocation.

Anyway, I will re-introduce the raise_on_error config option that was in ipytest until some time ago and return the error code. Then you could handle your issue with ipytest.autoconfig(raise_on_error=True).

[chmp]

@itepifanio I released ipytest==0.12.0b1 with raise_on_error and the return code in run. It would be great if you could give it a spin and report back whether it works for your needs.

[itepifanio]

@chmp it works very nice, thanks!

[chmp]

Prefect. Thanks for the feedback. I will try out the pre-release myself for a bit and then create a release in the next days.

[chmp]

Released as https://pypi.org/project/ipytest/0.12.0/