search

nolanlawson / blob-util

Cross-browser utils for working with binary Blobs
https://nolanlawson.github.io/blob-util
Apache License 2.0
503 stars 45 forks source link

Question: store data in browser file cache #59

Closed shaunc closed 4 years ago

shaunc commented 5 years ago

Thank you for this library. I am writing a data visualization tool. I would like to cache data (stored in flatbuffers ... so essentially an ArrayBuffer), but there could be several megabytes -- possibly more than I can fit in localForage. I'm wondering if this library can be used to store data in the browser history file cache as images (using hidden image tags, for instance), and use etags for invalidation.

It would seem I can convert an image into a blob and from there into an ArrayBuffer (does this duplicate storage or is the array buffer a view of the image?) Are there any gotchas that I'm not thinking of?

nolanlawson commented 4 years ago

Hi @shaunc, typically you can store Blobs or ArrayBuffers directly in IndexedDB. You might take a look at something like idb-keyval: https://github.com/jakearchibald/idb-keyval. Typically you shouldn't even need to convert before storing, especially if the original object you're working with is a File (which implements the Blob interface). Good luck!

shaunc commented 4 years ago

Nolan, thanks for the reply. I was considering storing as files to get around the size limitations on indexdb (the data is historical timeseriies for many different datapoints -- I'm hoping to cache whichever the user requests, the more cached the better) . Do you know if caching as images will work... and will it duplicate memory use (I guess this might depend on the browser)?

nolanlawson commented 4 years ago

Size limitations in IndexedDB also typically apply to any other kind of origin storage (Cache API, localStorage, etc.). If you're hitting those kind of limits, you may need to use the quota/storage API to request additional quota. I admit I'm less familiar with this area, but here's a link that may help: https://developer.mozilla.org/en-US/docs/Web/API/Storage_API

Typically for caching images, you would want to use the Cache API in a Service Worker. You may also want to optimize your images before sending them to the user (check out ImageOptim) which may help with taking up too much space. Good luck!