georgestagg
commented
1 year ago Copying a comment here with an update on the current situation,
For webR in particular a persistent VFS is tricker than at first glance. We cannot immediately use solutions that work well for other WASM projects because we use a blocking worker thread to maintain our place in the stack while waiting for input for R. Due to this we cannot easily yield to the event loop in the worker thread[1].
From what I currently understand there are several options available, but each have their own drawbacks for us,
- localStorage and sessionStorage are unavailable in web workers, where we perform our R computation.
- "OPFS" looks promising, but at the time of writing is only available in bleeding-edge Chromium browsers.
- Emscripten's new "wasmFS" also looks promising but is currently Work in Progress.
- The IndexedDB API (used by Emscripten's IDBFS driver) is asynchronous only. Since we block in the worker thread we're not able to use this API.
- Service workers can be (ab)used to persist data, but only a single service worker can be active on the page at a time.
I think the future of persistent storage in webR is going to be based around synchronous OPFS or wasmFS API, once they are ready and supported by more browsers. In the meantime, solving this issue will probably involve significant new infrastructure based around passing messages to the main thread to persist data.
[1] Asyncify would allow us to yield, but it does not work well for us due to the way the R interpreter works and the large asyncify overhead induced.
gregvolny
aehlke
Perhaps by mounting an Emscripten
IDBFSfilesystem somewhere like/storage.We will need to think about long term storage implications. Browser localStorage will work in the short term but could be cleared away at any moment.