Implement Save For Later: save page contents, screenshot, and metadata

[victorporof]

What a 'snapshot' actually is isn't completely defined yet, but for now readability text, marked-up text and a screenshot of the page are good bets. Currently we just store readability text.

[victorporof]

This functionality would, for now, be different than "bookmarking" (or "starring") a page.

[rnewman]

This is probably a meta bug at this point. I think this includes:

• [ ] Saving page content as HTML/webarchive.
• [ ] Taking a whole-page screenshot.
• [ ] Adding metadata to the DB. This'll be quite close to starring, but pages can be saved multiple times. We'll want to track keywords, titles, pointers to the content we've saved, the timestamp, etc. etc.
  • [ ] https://github.com/mozilla/tofino/issues/567 tracks surfacing <meta> elements and events.
• [ ] UI to trigger the action.

Follow-up bugs:

• [ ] Retrieval. Ambient ("saved versions of this page"), direct (some kind of list UI), other?
• [ ] Management. Deletion, in particular.
• [ ] Surfacing the distinction between saved and starred pages in the awesomebar.

[bgrins]

I was wondering about saving as HTML/webarchive. I couldn't find any feature directly on the webview to fetch as a webarchive file. But, it looks like we can save the page as MHTML which looks really promising: https://github.com/electron/electron/blob/master/docs/api/web-contents.md#webcontentssavepagefullpath-savetype-callback. Or as PDF: https://github.com/electron/electron/blob/master/docs/api/web-view-tag.md#webviewprinttopdfoptions-callback

[bgrins]

Sample code for saving the current webview as an MHTML file:

$0.getWebContents().savePage("/tmp/test.mhtml", 'MHTML', (error) => {
  if (!error) { console.log('Save page successfully'); }
});

One challenge is that it saves directly on the hard drive, while we might instead want the rendering process to fetch it as a buffer that could be sent to the UA service for saving. But we might need to directly save this in the profile folder, keyed on a page's unique ID. This would complicate sharing this data between two clients connected to same UA service.

[bgrins]

We have the capturePage API for getting a screenshot: https://github.com/electron/electron/blob/master/docs/api/browser-window.md#wincapturepagerect-callback

[bgrins]

My first thought is let's add a button next to the bookmark start to trigger the save action. @phlsa any opinion about how to surface this and what it would look like?

[bgrins]

@rnewman do we already have an ID that we could key these saves off of? A sessionID seems too wide, since you could have multiple saves per session. We could key on URL as in bookmarking, although the contents could be different for the same URL at different times (if the page has been updated, or if you were logged in at one point and not the other).

[rnewman]

For the moment I'm totally satisfied with saving on disk on the local machine. I'd much rather have Dropbox solve file storage and replication than us. We can stuff the path, file hash, timestamps, file size, etc. into storage, of course.

[victorporof]

@bgrins nice work researching this.

[rnewman]

To reply to comments since that email response: I'd store the saved page on disk and then make reference to it from the DB, rather than the other way 'round.

A save event would be something like:

id: long
session: long
ts: long
path(s): string, relative to storage base
place ID: long
page ID: if we just snapshotted reader content for indexing, refer to it here
file metadata, if you wish

Yes, this means we can have dangling pointers if the files go missing. That's OK.

[rnewman]

I'm happy to tackle the teensy storage bit in code if you like; fling an issue at me when you're ready for that bit.

[phlsa]

We'll be reviewing some ideas around saving later today. As it looks right now, we could probably just wire up the star button to this new saving functionality. I don't think that we'll end up in a world where we will have both classic bookmarks and saving.

[victorporof]

@phlsa @rnewman I thought that's exactly what we were trying to avoid? "Starring" is something different than "saving".

[rnewman]

By "classic bookmarks", I understand:

• Hierarchical
• Named, tagged, etc.
• Organized
• Potentially duplicated

We observed that that single piece of functionality is currently bent towards three or more distinct purposes:

• Things you want to save and return to. (a recipe, a poem, a photo). People also use screenshots/File>Save As/Evernote etc. for this, because the browser doesn't offer true offline/permanent storage.
• Places that start a task. (facebook.com, expedia.com)
• Non-functional bookmarks: lifting the mental weight of closing a tab; indicating that you like something; etc.

My understanding of our discussions last week is that we acknowledge and intend to support the first two tasks through two separate metaphors:

• Saving, which stores timestamped content/metadata/etc. to allow for return, rediscovery, offlining, and total confidence that you'll never lose content.
• Starring, which is a simple indicator that you want to go back to a page later.

I don't want to "Save" facebook.com — I don't care about the content, I want the entry point. I don't want to "Star" my boarding pass — I need it offline. Those are two different things.

Do you concur, @phlsa?

[phlsa]

Yeah, that's roughly what I was thinking. We started to work through the information architecture here and hope to have a coherent recommendation to explore/test some time next week.

[victorporof]

Oops this shouldn't be a part of the MVP.