In notebook, stream output, allow interrupt.

[switham]

When a user-python process in the notebook takes a while,

• Show output as it's produced.
• Show the fact that it's still working.
• Allow the process to be interrupted.
• When it's finished working, show that.

This requires processes or threads on the server. (At least one user-python thread, plus the server itself.) Also requires Ajax (xml-http-request) stuff. I don't even know whether wsgiref.simple_server supports requests in parallel. Not that that's absolutely necessary: the server can return from running a Python block quickly after having launched a separate thread or process to do the actual work.

Anything I do requires some non-trivial (to me) javascript to periodically poll the server for more output. I am imagining polling once a second, not very low latency.

I don't know whether I want to import a complete pseudo-sockets library. It's good not to reinvent the (two-sided Python/Javascript) wheel, but it would have to be pure python on the server, and I want it not too bulky. Assuming multiple open notebooks, what I need here does require streams open in parallel, but only coming from server to client.

When one python block's code is still running, the notebook shouldn't let you start another running.

The idea that you might have multiple notebooks open at once is [https://github.com/switham/pyinthephone/issues/09].

[switham]

The Most Crappy Minimum-Change Way

The way the code currently works is

• The server keeps all the user code blocks and their outputs in, let's say the "data list".
• The code entry area is a textarea in a form, and the "Run" button is its submit button, which sends a POST request with the user's chunk of Python code.
• The server runs the code within the same thread and process as the http request handler itself, using exec() with a supplied globals dictionary, and redirecting stdout and stderr to stringIO buffers.
• The server waits for the user code block to finish (i.e. just calls exec(), which is blocking).
• The server adds the code block and outputs, as well as a new empty textarea form, to the notebook data list.
• The server regenerates the whole HTML page from the data list and returns that new page as the response to the POST. There is an anchor within the page where it's supposed to scroll to (but see https://github.com/switham/pyinthephone/issues/17 ).

So, here is what seems to me a minimum modification. The idea that the web server should always return fast, and the idea of putting the user output in actual files, come from David Baumgold @singingwolfboy .

• Redirect stdout and stderr to actual files in, say, the data directory, or a tmp directory. Maybe thru a wrapper that forces flushing on every print.
• Server uses the "threading" module, so that the globals and code are all set up and run the same way (modified globals available to the server for the next user code "Run" request) except in a thread.
• Server waits for either process completion or one second (whichever's earliest) and replaces (see below) the output section under this code block with the whole contents of the output files. See [https://github.com/switham/pyinthephone/issues/31] about handling stderr, but don't solve that problem for now.
• Server regenerates the whole HTML page, except now there's one variation for thread done, and one variation for thread still running.
• The still-running version... well this is really minimal:
  • Says "running" somewhere.
  • Has a completely manual "see more" button, and a stop button! "See more" sends a regular "GET" request! (as does "stop").
• No automatic polling for output. Two problems with that are bypassed by postponing that for now:
  • How to prevent the update from getting into a fight with the user about what place in the output to scroll to, given that we're still refreshing the whole page and scrolling to an #anchor. Making it manual means this scrolling only happens when you press "more", and to the place where the "more" button just was.
  • Where to put the "stop" button (or what key should it be) so that it doesn't keep scrolling out of sight when the user wants to press it.
• The "see more" handler in the server is pretty much the same as the end of the POST handler: it replaces the outputs in the data list with the files' whole contents, then regenerates the whole HTML page in either done or not-done mode.