nteract / testbook

🧪 📗 Unit test your Jupyter Notebooks the right way
https://testbook.readthedocs.io
BSD 3-Clause "New" or "Revised" License
416 stars 37 forks source link

Entire notebook executed before the specific method #131

Closed aurnoi1 closed 3 years ago

aurnoi1 commented 3 years ago

The problem

Given I have a cell containing an input request:

number = input("Enter your number please")

And another cell with a function I would like to unit test:

def add_one(number):
    return number + 1

When executing the unit test like this (I took out some import related to relative path):

from typing import Any
from testbook import testbook

@testbook('src/notebooks/api/sub-domain/example2.ipynb',
          execute=True)
def test_add_one_returns_value_add_one(
        tb: Any) -> None:
    add_one = tb.get("add_one")
    actual: int = add_one(2)
    assert actual == 3

Then the testbook is executed before the unit test block And error is raised: "raw_input was called, but this frontend does not support input requests. StdinNotImplementedError: raw_input was called, but this frontend does not support input requests."

Environment:

testbook: 0.4.2

devcontainer: mcr.microsoft.com/vscode/devcontainers/python:0-3.9

jupyter core : 4.7.1 jupyter-notebook : not installed qtconsole : not installed ipython : 7.24.1 ipykernel : 5.5.5 jupyter client : 6.1.12 jupyter lab : not installed nbconvert : not installed ipywidgets : not installed nbformat : 5.1.3 traitlets : 5.0.5

Information

I just start to develop in Python so I may miss something here. I want to use this work to do some interactive demo with coworkers. There "input" request is not related to the error since any other code block is also executed.

Workaround

Add a condition before each block to prevent execution:

python

if interactive: number = input("Enter your number please")

Thanks!

rohitsanj commented 3 years ago

Hey, thanks for raising this issue. Your workaround would work fine, but testbook allows for specific cell execution before a unit test. If you could tag the cell containing the add_one function as 'cell_to_test' or something, you can then write the test like so:

@testbook('src/notebooks/api/sub-domain/example2.ipynb', execute=['some_imports_cell', 'cell_to_test',])
def test_add_one_returns_value_add_one(tb: Any) -> None:
    add_one = tb.get("add_one")
    actual: int = add_one(2)
    assert actual == 3

Read more about using execute kwarg here. Also read more about tagging in Jupyter notebook over here.

PS: You can also pass in the cell index of the cell (starting from 0), but this is not reliable and hence not documented. Using tags is better because the cell can move around but still have the same tags, but it wont have the same index when moved.

Let me know if this helps. Thanks! :D

rohitsanj commented 3 years ago

If you'd like to be invited to the nteract Slack workspace, drop me an email at hi@rohitsanjay.com.

aurnoi1 commented 3 years ago

Hi @rohitsanj ,

I confirm it works using tags. I have tagged a previous cells containing all import and the cell to test as you recommended. The cell containing the Input function is not called as expected.

Unfortunately I am using VSCode Jupyter extension and it does not have tags feature yet.

I have edited the json content to be able to have the tag, but running manually the notebook in vscode seems to reset the metadata.

So, I have used cell index instead and it works well too. Given the problem with tags in vscode extension, it is less work to use index. Please do not remove it in the next releases!!! :)

Thanks for your help!