search

DallasHoff / sqlocal

SQLocal makes it easy to run SQLite3 in the browser, backed by the origin private file system.
https://sqlocal.dallashoffman.com
MIT License
322 stars 15 forks source link

Database maximum size #18

Closed MirjamElad closed 7 months ago

MirjamElad commented 7 months ago

Hello and thank you for this nice library.

I understand this is supposed to be run in the context of a web browser. With that in mind, how much data can the "DB" hold in practice. Meaning what is the maximum data size you effectively tested this library with while keeping responsive and practical.

Thank you again and best wishes!

MirjamElad commented 7 months ago

A side note to my question above. I might be tempted to use sqlocal with the socket runtime (see here: https://github.com/socketsupply/socket/). With the socket runtime, the app would be a native app (it uses the native webview not the browser) and hence the storage space it uses will not be subject to be reclaimed by the operating system (think in the context of a mobile browser that uses storage such as IndexedDB or OPFS). All of that is to say, the database "managed" by sqlocal can grow and have real durability quarantees. For such scenario, knowing how large of a DB sqlocal can effectively handle becomes very important to test/know

DallasHoff commented 7 months ago

SQLite can handle databases of enormous sizes (many, many terabytes), and SQLocal does nothing that would limit this. The only thing really limiting the size of the database is the limits imposed on the Origin Private File System API by the browser. These limits vary between browsers, but if your app requests persistent storage, you should be able to store a database around 50% of the size of the client device's disk size, and the browser will not delete it. MDN has a good write-up about how this works. I recommend reading it: https://developer.mozilla.org/en-US/docs/Web/API/Storage_API/Storage_quotas_and_eviction_criteria

In practice, you should not have to worry about being constrained by the limit. The limits are much higher than you should need for pretty much any use case. The only case you really need to handle is checking that the user's disk is not nearing capacity. You can use the Storage API for this.

With the socket runtime, the app would be a native app (it uses the native webview not the browser) and hence the storage space it uses will not be subject to be reclaimed by the operating system

As far as I know, this and frameworks like it (Capacitor, Tauri) share the same behaviors as the operating system's default browser (Edge on Windows, Safari on macOS/iOS, Chrome on Android). They create native apps that wrap a browser instance. So you should expect the Storage API for these apps to work the same as the default browser.

MirjamElad commented 7 months ago

Thank you for the reply.

(Capacitor, Tauri) share the same behaviors as the operating system's default browser That was my understanding but I ve been told otherwise. Not sure 100% as I did not find clear information though!

Anyone tested SQLocale on big data. 100 MB and up to 1 GB?