computationalmodelling / nbval

A py.test plugin to validate Jupyter notebooks
Other
438 stars 50 forks source link

Documentation about skipping certain output types do not work #168

Closed tlvu closed 2 years ago

tlvu commented 3 years ago

I was following the following doc https://nbval.readthedocs.io/en/latest/index.html?highlight=skip#Skipping-certain-output-types and added

def pytest_collectstart(collector):
    collector.skip_compare += 'text/html', 'application/javascript',

to my conftest.py (see commit https://github.com/Ouranosinc/PAVICS-e2e-workflow-tests/commit/36d2fb42fcea81e2d5b14034d10298ec51bb7d08 in PR https://github.com/Ouranosinc/PAVICS-e2e-workflow-tests/pull/73)

But I ended up with this error:

py.test --nbval notebooks/hummingbird.ipynb --sanitize-with notebooks/output-sanitize.cfg
================================================================ test session starts =================================================================
platform linux -- Python 3.7.10, pytest-6.2.3, py-1.10.0, pluggy-0.13.1
rootdir: /repos/PAVICS-e2e-workflow-tests
plugins: tornasync-0.6.0.post2, anyio-2.2.0, nbval-0.9.6, dash-1.20.0
collected 0 items                                                                                                                                    
INTERNALERROR> Traceback (most recent call last):
INTERNALERROR>   File "/opt/conda/envs/birdy/lib/python3.7/site-packages/_pytest/main.py", line 269, in wrap_session
INTERNALERROR>     session.exitstatus = doit(config, session) or 0
INTERNALERROR>   File "/opt/conda/envs/birdy/lib/python3.7/site-packages/_pytest/main.py", line 322, in _main
INTERNALERROR>     config.hook.pytest_collection(session=session)
INTERNALERROR>   File "/opt/conda/envs/birdy/lib/python3.7/site-packages/pluggy/hooks.py", line 286, in __call__
INTERNALERROR>     return self._hookexec(self, self.get_hookimpls(), kwargs)
INTERNALERROR>   File "/opt/conda/envs/birdy/lib/python3.7/site-packages/pluggy/manager.py", line 93, in _hookexec
INTERNALERROR>     return self._inner_hookexec(hook, methods, kwargs)
INTERNALERROR>   File "/opt/conda/envs/birdy/lib/python3.7/site-packages/pluggy/manager.py", line 87, in <lambda>
INTERNALERROR>     firstresult=hook.spec.opts.get("firstresult") if hook.spec else False,
INTERNALERROR>   File "/opt/conda/envs/birdy/lib/python3.7/site-packages/pluggy/callers.py", line 208, in _multicall
INTERNALERROR>     return outcome.get_result()
INTERNALERROR>   File "/opt/conda/envs/birdy/lib/python3.7/site-packages/pluggy/callers.py", line 80, in get_result
INTERNALERROR>     raise ex[1].with_traceback(ex[2])
INTERNALERROR>   File "/opt/conda/envs/birdy/lib/python3.7/site-packages/pluggy/callers.py", line 187, in _multicall
INTERNALERROR>     res = hook_impl.function(*args)
INTERNALERROR>   File "/opt/conda/envs/birdy/lib/python3.7/site-packages/_pytest/main.py", line 333, in pytest_collection
INTERNALERROR>     session.perform_collect()
INTERNALERROR>   File "/opt/conda/envs/birdy/lib/python3.7/site-packages/_pytest/main.py", line 620, in perform_collect
INTERNALERROR>     rep = collect_one_node(self)
INTERNALERROR>   File "/opt/conda/envs/birdy/lib/python3.7/site-packages/_pytest/runner.py", line 457, in collect_one_node
INTERNALERROR>     ihook.pytest_collectstart(collector=collector)
INTERNALERROR>   File "/opt/conda/envs/birdy/lib/python3.7/site-packages/pluggy/hooks.py", line 286, in __call__
INTERNALERROR>     return self._hookexec(self, self.get_hookimpls(), kwargs)
INTERNALERROR>   File "/opt/conda/envs/birdy/lib/python3.7/site-packages/pluggy/manager.py", line 93, in _hookexec
INTERNALERROR>     return self._inner_hookexec(hook, methods, kwargs)
INTERNALERROR>   File "/opt/conda/envs/birdy/lib/python3.7/site-packages/pluggy/manager.py", line 87, in <lambda>
INTERNALERROR>     firstresult=hook.spec.opts.get("firstresult") if hook.spec else False,
INTERNALERROR>   File "/opt/conda/envs/birdy/lib/python3.7/site-packages/pluggy/callers.py", line 208, in _multicall
INTERNALERROR>     return outcome.get_result()
INTERNALERROR>   File "/opt/conda/envs/birdy/lib/python3.7/site-packages/pluggy/callers.py", line 80, in get_result
INTERNALERROR>     raise ex[1].with_traceback(ex[2])
INTERNALERROR>   File "/opt/conda/envs/birdy/lib/python3.7/site-packages/pluggy/callers.py", line 187, in _multicall
INTERNALERROR>     res = hook_impl.function(*args)
INTERNALERROR>   File "/repos/PAVICS-e2e-workflow-tests/conftest.py", line 2, in pytest_collectstart
INTERNALERROR>     collector.skip_compare += 'text/html', 'application/javascript',
INTERNALERROR> AttributeError: 'Session' object has no attribute 'skip_compare'

=============================================================== no tests ran in 0.01s ================================================================

Which is fixed by adding this line https://github.com/Ouranosinc/PAVICS-e2e-workflow-tests/blob/e31a2a12ad9a6210c785f5e60f8e8b49ce487c17/conftest.py#L2

So I think the documentation should be updated.