jvilk / BrowserFS

BrowserFS is an in-browser filesystem that emulates the Node JS filesystem API and supports storing and retrieving files from various backends.
Other
3.07k stars 218 forks source link

Change the mutex for the LockedFS to be file-based #287

Closed d3lm closed 1 year ago

d3lm commented 4 years ago

We are currently using BrowserFS for our project and so far it works well. Great work! One thing that we have encountered though is that we use it in a multi-process / thread environment where the FS (an OverlayFS) lives in a dedicated worker and other web workers can perform fs operations by communicating to the fs worker. This means that technically multiple workers could call at the same time for example readFile but the LockedFS uses a mutex that doesn't differentiate between files it's operating on. This means it's a bit of a bottle neck because if you call readFile on 3 different files for instance, one gets in and 2 have to wait until the first call is resolved. I can imagine that BrowserFS was not designed to be used in a multi process environment but even in a single thread this would hold true for async calls.

Another thing is that, when an async call is being processed, any sync call that comes from another worker will just fail, because for the LockedFS again, it only checks if the lock is free, and if not, it will throw an error. Wouldn't it make sense to also use a mutex for the sync calls? So that sync calls get queued as well and they don't just fail. I do see that a sync call blocks async calls.

Curious to hear your thoughts?

d3lm commented 4 years ago

Or maybe another question, what is the lock for in the LockedFS? I mean the kernal (Linux for example) also doesn't use locks for fs calls and kind of allows file corruptions. However, metadata has to be protected to avoid a corrupted overall fs state.

james-pre commented 1 year ago

Done! cc1e648