jvilk / BrowserFS

BrowserFS is an in-browser filesystem that emulates the Node JS filesystem API and supports storing and retrieving files from various backends.
Other
3.06k stars 216 forks source link

AsyncMirror with IndexedDB does not seem to sync whening using Pyodide #322

Closed vrthra closed 1 year ago

vrthra commented 2 years ago

I am trying to use BrowserFS with pyodide. My configuration is as follows

BrowserFS.configure({
  fs: "AsyncMirror",
  options: {
    sync: { fs: "InMemory" },
    async: {
      fs: "IndexedDB",
      options: { storeName: "pythoninstall", },
    }
    // cacheSize: 100 MB is default inode cache.
  }
}, function(e) {
if (e) { throw e; }
languagePluginLoader.then(() => {
  // https://github.com/pyodide/pyodide/issues/613
  let FS = pyodide._module.FS;
  let PATH = pyodide._module.PATH;
  // Create an Emscripten interface for the BrowserFS
  var BFS = new BrowserFS.EmscriptenFS(FS, PATH);
  // Create mount point in Emscripten FS
  //FS.createFolder(FS.root, 'data', true, true);
  FS.createPath(FS.root, 'data', true, true);
  // Mount BrowserFS into Emscripten FS
  FS.mount(BFS, {root: '/'}, '/data');
...
})

Next, within pyodide, I try this in one window.

with open('/data/x.txt', 'w+') as f:
     f.write('Hi there\n')

This works in the same window

with open('/data/verify.txt') as f:
     print(f.read())

But not if I use a different window.

According to the AsyncMirror docs, both backends should be kept in sync. However, the write operation does not seem to be synced to IndexDB even after some time has passed.

james-pre commented 1 year ago

@vrthra IndexedDB (even when using AsyncMirror) is still asynchronous across different frames (windows). You should instead use a cross-frame synchronous backend instead (e.g. LocalStorageFS) as the sync backend for AsyncMirror (instead of something like InMemory, which is isolated to a window).