Exclusive write locks (related to concurrent io)

[jlongster]

In the "concurrent io" section:

The current prototype of Storage Foundation API allows any file to be opened only once. In particular, this means that files can not be shared between tabs. Allowing a file to be opened multiple times requires significant overhead to prevent concurrent, unsafe writes to the same file. Our current approach is therefore quite safe yet restrictive. We may revisit this decision based on developer feedback.

That is a huge drawback of the API, and renders most important use cases much less useful.

I just released a project of mine called "absurd-sql": https://github.com/jlongster/absurd-sql. I wrote a blog post explaining it here: https://jlongster.com/future-sql-web. It's a virtual filesystem backend for sql.js that allows databases to live in persistent storage. Currently, it uses IndexedDB because that's the only cross-browser persistence API that works.

One of the crucial things that IDB gives us is locking in the form of readwrite transactions: once we have that transaction open, we know we are the only ones that can write to it.

If things like database are one of the target uses for this API (which is what I've been told from talking with google devrel), we definitely need file locking. It's crippling to allow apps to only run in one single tab.

Now, we don't need enforced file locking. One thing that could reduce the overhead is implementing something like posix advisory locks. We just need a signal to other processes that we have locked the file. Basically we just need a atomic "check-and-swap" operation which lets us just if we've successfully grabbed the lock.

[rstz]

Thank you for the feedback and your interest in Storage Foundation API! absurd-sql looks really cool, and I'll check out how you implemented it soon. Databases are definitely a use case for this API.

We rely on exclusive locks for files for performance reasons; they both heavily reduce the implementation complexity and prevents us from having to manage expensive communications between tabs before accessing a file.

When we considered the need for more flexible locking, our hope was the Web Locks API would provide the right primitive to outsource more complicated access control to it. When applied to Storage Foundation, after taking a lock, a file can be opened, written to, and then closed again before releasing the lock. Of course, closing and re-opening a file very often would have some performance impact; a performance-sensitive application should invest in more sophisticated patterns.

More recently, we've worked on adding some functionality of Storage Foundation API to the origin-private file system of the File System Access API. We intend to move forward with the File System Access API in the foreseeable future; it’s probably useful to focus on the corresponding discussions in issue in FSA’s explainer, such as WICG/file-system-access#325

[jlongster]

Sounds great. Makes a lot of sense to focus on the File System Access API, and if that's where the focus will be feel free to close this issue and we can discuss more in the issue you linked (the other one I created).

fwiw it's totally fine that these features are only available to the OPFS for now. While working with local files is nice, imho it should be the highest important to have this feature at all, and if it makes it easier to only focus on OPFS that makes sense. As long as the data is persistent (browsers won't delete it) I don't think it's a huge difference.