Notebook stuck in run mode

[tritemio]

I have a weird issue which involves the notebook. The notebook gets stuck during the execution as some random position, the CPU becomes idle and the computation stops, while the notebooks shows the the In[*] prompt on the remaining cells. The system is windows x64, conda, python 3.4 with notebook 4.0.6 and nbconvert 4.1.

I have a set of notebooks used to analyze a dataset. These notebooks are run in sequence by a "master" notebook.

To run a notebook from the master notebook I use a function (run_notebook()) which does:

nb = nbformat.read(nb_name_full, as_version=4)
ep = ExecutePreprocessor(timeout = 3600)
out = ep.preprocess(nb, {'metadata': {'path': './'}})
nbformat.write(nb, out_nb_name)
display(FileLink(out_nb_name))

Some notebooks take also input parameters which I add to the first cell before execution. In this case, I use a modified function (run_notebook_template()) which adds a cell with the parameters as follows:

nb = nbformat.read(nb_name_full, as_version=4)
nb['cells'].insert(1, nbformat.v4.new_code_cell('data_id = "%s"' % data_id))
nb['cells'].insert(1, nbformat.v4.new_code_cell('ph_sel_name = "%s"' % ph_sel))

The complete functions are here.

Executing the master notebook with Run-All, or cell-by-cell, causes the execution to stop in some random cell calling run_notebook_template() (the parametric runner).

No messages are shown in the console (even in debug mode) except for autosaving messages. Only occasionally (i.e. after letting the notebook run overnight) I get this:

[IPKernelApp] WARNING | Timeout waiting for IOPub output

Let me know if you need more info.

[minrk]

Do you have a complete example, including notebooks, to reproduce the issue? My guess is it's race conditions or something like that in the execute preprocessor, or possibly failure to notice and propagate errors.

[tritemio]

@minrk, after a recent update (probably tornado 4.2 -> 4.3?) I see this issue much less. I'll report any updates.

[tritemio]

@minrk, for the record, I tried to remove all the exception handling in the functions that run the notebooks (i.e. run_notebook, see link above) and I don't get any exception.

[jdfreder]

@tritemio this is also a symptom of the frontend throwing an unhandled error. Can you open the web developer console of your web browser, and see if you see any errors there?

[tritemio]

@jdfreder, @minrk I still have this problem with notebook 4.2rc1 (nbconvert 4.2.dev).

Now I have created a stand-alone example of the master notebook executing a template notebook which you can find here.

I've tested it on OS X and Windows and I only have the problem on Windows. The web console does not show any errors, as there is no error/message in the terminal used to launch jupyter notebook.

The issue is not deterministic. I need to execute the master notebook several times until it gets "stuck". But I definitely can reproduce it within a few minutes.

[tritemio]

FYI, I have reproduced the issue on a second windows machine with notebook 4.0.6 and nbconvert 4.1.

[willingc]

@tritemio Thanks for reporting that you were able to reproduce.

@minrk @jdfreder I will try to reproduce on a Windows 10 machine later this weekend. Will let you know what I uncover.

[minrk]

Thanks for more details. The circumstances are giving me the suspicion that we are doing something not quite right in our zmq sockets, which can be a pain to debug, especially when it seems to only be a problem on Windows. I'll try to find time to dig into it.

[tritemio]

@minrk @willingc let me know if you are able to reproduce the issue. The two machines I tested with both run Windows 7 x64 and Anaconda and spinning-disk hardrive. On these machines the cycle of executing and saving a trivial notebook takes > 2 seconds. It's faster on OS X (and I suspect linux, from previous experience). Maybe this additional delay exposes a race condition?

[YoavEtzioni]

Hi

I still get stuck like this from time to time, and it is hard to reproduce. Any suggestion for remedy?

[huggybearikf]

I've been experiencing the same error, on two different installations on different Windows computers. I tend to find that the error occurs most often when I try to peak Pandas DataFrames in Notebooks or other operations that should be fast, which supports the "race condition" hypothesis.

For those encountering this issue, sometimes I can interrupt the program by going to Comman Prompt and Ctrl-c once to force keyboard interrupt. But whether the final cell gets executed appears to be non-deterministic.

Alternatively, if its an option, install jupyter in the Windows 10 linux shell (I used Ubuntu) and run it through there. You can still use the interface through Chrome, and I've tried it there and have not had an issue, but it does add quite a few layers of pathing.

[huggybearikf]

Also, I've not had the same issue running Hydrogen in Atom through my Windows kernel. If you are working on Jupyter for prototyping, this is another alternative.

[nmaiorana]

I'm having the same issue with Microsoft Surface Pro 4 running Windows 10. I execute a cell and it just hangs. The Notebooks get saved automatically, so I know the kernel is not completely hosed.

[jknorvik]

I get the same issue with notebooks on Google Cloud DataLab after converting a pandas df to a pyspark.sql.dataframe and then trying to do anything with the latter - count(), show(), select().distinct() ... Using Chrome on Windows 10 to access DataLab

[tariksalay]

still having the same issue

[wikd13]

Issue still exists.

[gd-arch]

jupyter notebook gets stuck if i try to run n=int(input()) and press shift +enter then without entering an input if again press shift +enter by selecting the code contents the notebook seems to get stuck.i was wondering if this behavior is normal.I am using win 10