WebMemex / webmemex-extension

📇 Your digital memory extension, as a browser extension
https://webmemex.org
Other
208 stars 45 forks source link

Store snapshots as files in the user’s download folder #138

Closed Treora closed 4 years ago

Treora commented 5 years ago

It may be worth trying use the download folder as the main storage for snapshots, instead of IndexedDB. This because currently the extension feels like a silo that hides the snapshots in a secret corner of your disk, requiring you to ‘save as’ each page to do anything with it (sharing, editing, whatever). And also because, at least in my Firefox, IndexedDB has proved unreliable, having occasionally dropped all data of this extension after a browser upgrade or even without a clear reason.

Unfortunately, an extension does not have access to the contents of a downloaded file, which can make some features harder to provide. To still provide full text search, we could keep the snapshot’s text in IndexedDB; and then open a chosen snapshot using browser.downloads.open; but this would fail if the user has e.g. renamed the file; and it would be less than ideal if the default file handler for html files is not the browser running this extension; and it would be less than obvious why some can be searched, and some (e.g. those you copied from another device) cannot.

We could still show info about the snapshot (it’s date and original URL): when a locally stored snapshot file is opened in the browser (with file: URL), the extension could read the DOM and detect the <meta>&<link> tags that were added by freeze-dry, and thus recognise it as a snapshot. I am not sure however which browsers load and which refuse to load extensions in pages with file: URLs, but at least Firefox appears to load them nowadays.

I made a tiny step towards a possible switch of storage method in v0.3.2. It now provides the option to store every snapshot you make also in your downloads folder, as well as a button to download all previous snapshots at once:

screenshot of options

Treora commented 4 years ago

After observing users accidentally lose their data by moving to a new computer and understandably not moving their browser profile with them, I figured the default option should be to download snapshots as files.

In v0.4.0 this is implemented as described above, including the downsides:

To still provide full text search, we could keep the snapshot’s text in IndexedDB; and then open a chosen snapshot using browser.downloads.open; but this would fail if the user has e.g. renamed the file; and it would be less than ideal if the default file handler for html files is not the browser running this extension; and it would be less than obvious why some can be searched, and some (e.g. those you copied from another device) cannot.

The old behaviour is still available for those who want it. The code has become a little messy because of this change, but it seems not worth the effort reorganising it all now.

The idea of reading snapshot metadata from files is not yet implemented; moving that into a separate issue now.