add page about "How to test notebooks"

https://github.com/chmp/ipytest is one way to run pytest within jupyter notebooks.
pytest functions conventionally start with def test_.
There are many ways to make assertions of types and values in pytest test functions:
- x=None; if x is None: raise AssertionError(("x is None!", dict(x=x)))
- assert (the assert keyword)
- self.assert_(expr, msg) (or unittest.TestCase().assert_())
It's possible to call unittest assertion functions without a unittest.TestCase() in every Jupyter notebook input cell:


import unittest
Test = unittest.TestCase()

def test_testcase_methods_as_function_calls_that_raise_AssertionErrors():
  Test.assertTrue(True)
  Test.assertEqual("123", "123")

  with Test.assertRaises(ZeroDivisionError):
      1/0
  with Test.assertRaises(ValueError):
      int("0x0")

# a simple test runner that runs callables that start with `test_` (like pytest and iirc nose)
testscope = {**globals(), **locals()}
for name, testfunc in ((k,v) for k,v in testscope.items() if k.startswith('test_') and callable(v)):
    objstate = dict(name=name, func=testfunc, state='running')
    print(('running', objstate))
    try:
        objstate['output'] = testfunc()
        print(('completed', objstate))
    except KeyboardInterrupt:
        raise
    except Exception as exc:
        traceback.print_tb(exc.__traceback__)
        pass #

unittest has more helpful AssertionError messages:


>>> assert "123" == "1234"
AssertionError:
>>> unittest.TestCase().assertEqual("123", "1234")
AssertionError: '123' != '1234'

123
1234 ? +

.. note:: Because assert statements are optimized out with python -O and python -OO, test assertions that use the assert keyword will not run if the -O or -OO flags are passed to the python process that runs the tests.
- https://docs.python.org/3/using/cmdline.html#cmdoption-O
Shorter def test_()-style functions are more succint than class Test0(unittest.TestCase)-style functions
Shorter functions are easier to scroll through in Jupyter notebooks than TestCase subclasses (for simple assertions, at least)
Porting functions between TestCase subclasses with self. references to refactor is more work than copy/pasting def test_() functions, in the average case
Nose and pytest support functions named test_
- unittest can, too
pytest
- Source: https://github.com/pytest-dev/pytest
- Docs: https://docs.pytest.org/en/latest/
- Docs: https://docs.pytest.org/en/latest/contents.html
- Docs: https://docs.pytest.org/en/latest/reference/plugin_list.html
- Docs: https://docs.pytest.org/en/latest/how-to/unittest.html
- Pytest does run unittest.TestCase tests
- Docs: https://docs.pytest.org/en/latest/reference/fixtures.html
- Docs: https://docs.pytest.org/en/latest/how-to/fixtures.html
- Pytest has fixtures instead of unittest.TestCase's setUp and tearDown methods

sagemathinc / cocalc-doc

add page about "How to test notebooks" #40