nteract / testbook

🧪 📗 Unit test your Jupyter Notebooks the right way
https://testbook.readthedocs.io
BSD 3-Clause "New" or "Revised" License
416 stars 37 forks source link

`AttributeError: data` when repeatedly calling same function #148

Open The-Compiler opened 1 year ago

The-Compiler commented 1 year ago

With a trivial answer.ipynb like this:

```json { "cells": [ { "cell_type": "code", "execution_count": null, "id": "2382e31c-1117-46df-b981-81100e7f68f3", "metadata": {}, "outputs": [], "source": [ "def answer():\n", " return 42" ] } ], "metadata": { "kernelspec": { "display_name": "Python 3 (ipykernel)", "language": "python", "name": "python3" }, "language_info": { "codemirror_mode": { "name": "ipython", "version": 3 }, "file_extension": ".py", "mimetype": "text/x-python", "name": "python", "nbconvert_exporter": "python", "pygments_lexer": "ipython3", "version": "3.10.7" } }, "nbformat": 4, "nbformat_minor": 5 } ```

which simply contains a single cell with this code:

def answer():
    return 42

calling that function via testbook repeatedly (500+ times) will fail. Doing this:

from testbook import testbook

@testbook('answer.ipynb', execute=True)
def test_func(tb):
    answer = tb.get("answer")
    for _ in range(500):
        assert answer() == 42

if __name__ == "__main__":
    test_func()    # runnable with pure Python instead of pytest

results in:

Traceback (most recent call last):
  File "/home/florian/tmp/testbook/.venv/lib/python3.10/site-packages/nbformat/_struct.py", line 128, in __getattr__
    result = self[key]
KeyError: 'data'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/florian/tmp/testbook/test_answer.py", line 10, in <module>
    test_func()
  File "/home/florian/tmp/testbook/testbook/testbook.py", line 63, in wrapper
    return func(self.client, *args, **kwargs)
  File "/home/florian/tmp/testbook/test_answer.py", line 7, in test_func
    assert answer() == 42
  File "/home/florian/tmp/testbook/testbook/reference.py", line 85, in __call__
    return self.tb.value(code)
  File "/home/florian/tmp/testbook/testbook/client.py", line 302, in value
    return outputs[0].data['application/json']['value']
  File "/home/florian/tmp/testbook/.venv/lib/python3.10/site-packages/nbformat/_struct.py", line 130, in __getattr__
    raise AttributeError(key)
AttributeError: data

here:

https://github.com/nteract/testbook/blob/e2ae899930c30756e4b7db285d150dde1eacc9a1/testbook/client.py#L302

Adding a print(outputs) there reveals why:

[
    {
        'output_type': 'stream',
        'name': 'stderr',
        'text': "/home/florian/tmp/testbook/.venv/lib/python3.10/site-packages/IPython/core/displayhook.py:275: 
UserWarning: Output cache limit (currently 1000 entries) hit.\nFlushing oldest 200 entries.\n  warn('Output cache 
limit (currently {sz} entries) hit.\\n'\n"
    },
    {
        'output_type': 'execute_result',
        'metadata': {'application/json': {'expanded': False, 'root': 'root'}},
        'data': {'text/plain': '<IPython.core.display.JSON object>', 'application/json': {'value': 42}},
        'execution_count': 1003
    }
]

coming from IPython's displayhook.

Maybe instead of using outputs[0] there, testbook should find the first execute_result output or so? Though I'm also wondering if there's a way for testbook to prevent that warning altogether.