search

opencb / genome-maps

An open-source high-performance web-based HTML5 genome browser. Genome Maps can be customized and allow browsing user data such as BAM and VCF files among other formats.
http://www.genomemaps.org
GNU General Public License v2.0
39 stars 10 forks source link

rfc: how to invalidate indexDB #26

Open mbsimonovic opened 8 years ago

mbsimonovic commented 8 years ago

is it possible to invalidate all cached files when a new version gets deployed without having to change the dns?

jmmut commented 8 years ago

Did you mean IndexedDB? I think nowadays all browsers allow to see the caches. In firefox it's in developer tools > IndexedDB Browser

There you should be able to clear any DBs.

mbsimonovic commented 8 years ago

yes, indexedDB. What I mean is how can I as a developer for all caches to invalidate, once I deploy a new, lets say a bugfix release. I don't have an option to contact all users and tell them to "open dev tools, open resources, clear all indexedDB entries...", nor can I do it for them, it would just be ridiculous to invalidate thousands of web browser caches manually :)

jmmut commented 8 years ago

Ok, I think this is not very easy to do, depending on what level are you working. First of all, you need to detect that your deployed release is newer than the stored data, and only then clear the DBs. Take into account that after cleaning you should annotate (in some way) the DBs with some version to not erase them until the next release in the future.

To clear DBs, here is a method in indexedDB-store, which is the helper used in jsorolla to deal with IndexedDB. https://github.com/opencb/jsorolla/blob/master/src/lib/cache/indexedDB-store.js#L177

I don't know if you are using it behind the scenes or you are using directly IndexedDB, but in any case I think it will be useful.

mbsimonovic commented 8 years ago

sorry, the question wasn't clear, i was talking about genome-maps & jsrolla. how to clear indexedDB which they use to cache responses, once the data's changed on the server side. so the idea is to attach some kind of a build number, check on load in javascript, and if it's outdated, then call destroyDB?

mbsimonovic commented 8 years ago

how do you currently solve this? let's say you released genomemaps.org + cellbase this morning, and later today ensembl publishes a new release with some critical bug fixes, and you want to redeploy cellbase, but the users will have this data cached by now.

jmmut commented 8 years ago

I worked on these projects a year ago, on early stages of using IndexedDB, so I don't know how it's solved currently, sorry. At that moment it was not resolved. But if I were to implement it, yes, I would do it like you say on your previous message.

mbsimonovic commented 8 years ago

hm, would increasing iDBVersion make previous entries obsolete?

jmmut commented 8 years ago

I'm afraid not. If I remember well, IDB increases the version when adding objectStores to a DB (object stores are similar to tables in a regular SQL DB), so the version reflects a DB "schema" change, but new connections (with the new version) will still be able to access old data.

https://www.w3.org/TR/IndexedDB/#database-concept

https://developer.mozilla.org/en-US/docs/Web/API/IndexedDB_API/Basic_Concepts_Behind_IndexedDB

mbsimonovic commented 8 years ago

@jmmut just tried what you suggested, having a dedicated db to keep the last deployed version, and on upgrade to destroyDB, but these calls are async, so it's not going to work with the current CellBaseAdapter/FeatureChunkCache which are all synchronous:configureCache(), don't wait then call getData... there're some promise-based indexdb libs, https://www.npmjs.com/browse/keyword/indexeddb, would you consider replacing your implementation?