Piping not working on Python 3.5

[jluttine]

I'm trying to clean the test notebooks by using a pipe but it doesn't work on Python 3.5. I just get empty result:

$ cat tests/test_metadata.ipynb | nbstripout 

Running nbstripout tests/test_metadata.ipynb correctly cleans the notebook inplace.

I tried with Python 2.7 and then it outputs the normal cleaned notebook when piping. Any ideas what could be wrong?

$ pip freeze
decorator==4.0.11
docopt==0.6.2
docutils==0.13.1
ipython==6.0.0
ipython-genutils==0.2.0
jedi==0.10.2
jsonschema==2.5.1
jupyter-core==4.3.0
nbformat==4.3.0
nbstripout==0.3.0
path.py==10.1
pexpect==4.2.1
pickleshare==0.7.4
prompt-toolkit==1.0.14
ptyprocess==0.5
Pygments==2.2.0
simplegeneric==0.8.1
six==1.10.0
testpath==0.3
traitlets==4.3.2
wcwidth==0.1.6

[kynan]

I can't reproduce this. Have tried with both my system Python 3 and Anaconda Python 3. Could it be an issue with your terminal? Which platform are you on and which terminal are you using?

[jluttine]

Interesting. I'm using NixOS so that definitely can be a source of issues, though I don't know what it could be in this case. I'm using Bash on Konsole. I'd expect it is related to something that is done differently on Python 2 than on Python 3. Any ideas are welcome.

[jluttine]

And even more weird, this works on Python 3:

cat test_metadata.ipynb | python -m nbstripout

So it's just when it's used as a standalone executable in Python 3 that it doesn't handle piping properly. This is probably related to how NixOS wraps executables.

[kynan]

OK, good to know. Report back if you find out anything more.

[reidpr]

I can reproduce this on my Mac:

$ nbstripout < test_metadata.ipynb | wc
       0       0       0
$ python -m nbstripout < test_metadata.ipynb | wc
      88     167    1556

The problem appears to be buffering related. On one of my own notebooks, it appears that the output is empty unless it exceeds roughly 8192 bytes.

Adding output_stream.flush() after line 279 fixes the problem for me.

Version 0.3.0. Installed using conda.

[kynan]

@reidpr Thanks! I've pushed that fix. Can you try with master? I want to tag a new release.

[reidpr]

Happy to. Can you give me a brief recipe to get the executable nbstripout in a Git working directory? python setup.py build doesn't seem to make it, and python setup.py test complains pkg_resources.DistributionNotFound: The 'pytest' distribution was not found and is required by the application.

[kynan]

The 2 simplest options are:

• pip install git+https://github.com/kynan/nbstripout from anywhere
• python setup.py develop from a clone of this repo

[kynan]

@reidpr your fix will be released in 0.3.1

[kynan]

@jluttine @reidpr can you please test 0.3.1 and report if this is now fixed?

[reidpr]

This seems to work in 0.3.1.

Sorry I didn't test it earlier. It seemed like the test instructions required installing the version under test, so I kept putting it off. It would be great if everything could be tested from a Git checkout, without installing it.

[kynan]

The nbstripout script is just a setuptools entrypoint. To use that you do indeed need to install it. I recommend a python setup.py develop / pip install -e .. I'll add that to the README.

However you can just use python nbstripout.py ... if you want to test the version from your Git checkout.

[reidpr]

However you can just use python nbstripout.py ... if you want to test the version from your Git checkout.

Right, but in this case the bug only affected the nbstripout script.

Thanks for the fix, by the way. Pleasure working with you.