Add buffered stdin to accept terminal input whilst running WASM commands

[ianthomas23]

These are changes to support passing stdin from the terminal to a WASM command whilst it is running. It works with the corresponding changes in cockle in jupyterlite/cockle#23.

It uses SharedArrayBuffer to share information from stdin between the main worker (JupyterLite UI thread) and web worker (running the cockle shell), which allows the web worker to perform a synchronous blocking request to obtain the latest stdin. This cannot be performed via standard message passing between main and webworker via comlink as those are async and hence would not be received by the blocking WASM command whilst it is running.

Just before a WASM command is started, buffered stdin is enabled so that stdin messages from the frontend terminal are buffered in the main worker and the first character from stdin is inserted in the SharedArrayBuffer. If the webworker running the WASM command needs stdin it performs a synchronous Atomics.wait on the SharedArrayBuffer which blocks until such a character is available. One item (Int32) of the SharedArrayBuffer is used by the main worker to tell the webworker when a stdin character is available, and another item is used by webworker to tell the main worker that it has taken a character and it may load the next one if available. Stdin is finished when the shell receives an End-Of-Transmission character (ASCII 4) such as via Ctrl-D. Just after the WASM command finishes the buffered stdin is disabled and any characters already buffered are sent to the shell in the conventional way to avoid any data loss.

To try this out locally you need to use

jupyter lite serve --LiteBuildConfig.extra_http_headers=Cross-Origin-Embedder-Policy=require-corp --LiteBuildConfig.extra_http_headers=Cross-Origin-Opener-Policy=same-origin

to enable SharedArrayBuffer.

I am expecting many of the implementation details of the BufferedStdin classes to change when we have to support further functionality in the shell, so consider this just good enough for now.

These changes are really too complicated to be in this repo. Instead they and the webworker code should probably be move to cockle so keep this repo as simple as possible, relating only to integration with JupyterLite.

[ianthomas23]

I have been using wc to test this. Without any arguments it takes its input from stdin until Ctrl-D is received, when it calculates and shows the number of lines, words and characters. I have not included a screencast of this as it is really dull given that those stdin characters are purposefully not echoed back to stdout.

[ianthomas23]

@jtpio I presume that this will break the github pages deployment as some changes will be required to enable SharedArrayBuffer.

[jtpio]

@jtpio I presume that this will break the github pages deployment as some changes will be required to enable SharedArrayBuffer.

Yes. The documentation is still lacking some information about how to enable the SharedArrayBuffer on GitHub Pages: https://github.com/jupyterlite/jupyterlite/issues/1409

Using something like https://github.com/WebReflection/mini-coi

[ianthomas23]

This is on hold as there is an alternative implementation to investigate that does not use SharedArrayBuffer.

[ianthomas23]

I'm closing this as the WebWorker and SharedArrayBuffer code has been moved to cockle (jupyterlite/cockle#38). Changes will be needed here to use that, but in a separate PR.