RFC: CMS Autosave Feature

[unclecheese]

Picking this up from the original issue posted in 2012.

Abridged summary:

Web applications aren't as stable as desktop apps, and when they crash, it's very frustrating if you lose your content.

There are two ways this could be implemented: Auto-click the save button, or save content to localStorage.

I've had a number of discussions with core team members over how this would work, including @clarkepaul , @chillu , and @sminnee , and here are the options I see:

Option 1: Auto-save is purely a UI feature

With this approach, the problem we'd be addressing is simply the case where content authors lose changes when the browser tab closes, or the computer crashes, etc. It is member-specific, and form specific. It is entirely decoupled from the ORM.

Possible solution 1a: localStorage

localStorage saves a serialised form state, mapped to some hash of the form (e.g. `${window.location}-${formID || formAction})

Advantages

• Easiest implementation. Only touches client side code.
• No new tables, no updates to versioning.

Disadvantages

• It's not model or form-aware. Complex form fields like UploadField or TreeDropdownField, etc. wouldn't necessarily be able to persist their state in a serialised string. Assigning the value 87 to UploadField, for instance, is meaningless, and results in a partial loss of state.
• Not every saveInto or loadDataFrom call is the same. Some have specific implementations that the client can't possibly be aware of. The client side implementation of "saving" might be quite different from its server side counterpart, and users shouldn't have to know the difference.
• It doesn't persist across different workstations and browsers.

Possible solution 1b: A monolithic "autosave" table

We have a simple table, Autosave, that has a similar schema to what the above localStorage option would use. Fields would be something like FormHash, SerialisedFormState, MemberID.

Advantages

• By saving a serialised form state (e.g. just before saveInto(), we can guarantee that the state would be exactly the same, even when complex fields or custom data retrieval methods are used.
• Persists across workstations and browsers

Disadvantages

• More complex implementation. Would require new hooks around loadDataFrom() and saveInto() in various contexts. New APIs possible, but likely none broken.
• More network traffic, more perceived "slowness," particularly in the non-React areas.

Option 2: Auto-save is a versioning feature

This is a more fully integrated approach that puts us on the path to collaboration workflows in the spirit of Google Docs, et al. Here, we assume that every autosave is a version, albeit volatile, meaning it is member agnostic and model-centric.

Example: I get halfway through updating theJobDescription on an Employee object in ModelAdmin, and walk away. Another author then goes to the Pages section, and edits the Staff page, where Employees are displayed in a nested GridField. That author will now see my half-baked title (with an indication that the record has been retrieved from autosave) albeit in an entirely different context from my autosave in ModelAdmin.

Possible solution

Add a new field to the _verisions table for IsVolatile (bool). Client side events trigger a submit with a autosave=1 parameter, telling the form to either create a new version record with IsVolatile=1, or overwrite the latest version record meeting that criterion. On publish, all IsVolatile=1 versions are reset to permanent version records.

Advantages

• Of all the options, this is the closest analog to a true save, which means there would be the least chance of unpredictable state
• Conforms to the reasonable expectations content editors may have had instilled by collaboration tools like Google Docs
• One autosave applies to all contexts

Disadvantages

• Highly complex. Involves updating versioning with new APIs. We would now have to negotiate around where to display versions marked IsVolatile and where to filter them out. (Should a GridField summary view show the published/draft title, but then the edit view shows the autosaved title?)
• We would have to implement some kind of record-locking, which Hamish Friedlander articulately described as a difficult problem to solve
• One autosave applies to all contexts (yes, I see this as both an advantage and a disadvantage. The expectations will vary from user to user on whether the above Employee editing example feels right).
• More network traffic
• Race conditions around another user publishing while you're editing and blowing away your volatile version record (although record locking would likely block those conditions in most contexts).
• Should we forego record locking, merging content changes is a nightmare.

Considerations

• Do we deprecate the window.unload event that warns against loss of changes? I hope we can.
• We need UI to tell the user when a form was autosaved, and also to indicate when autosave state has been retrieved.

[chillu]

Not every saveInto or loadDataFrom call is the same. Some have specific implementations that the client can't possibly be aware of.

Can you give an example other than UploadField for this? I would assume that any data that's entered into a field will be represented in an internal form redux state. And the field can reinstate its accurate view based on that state. For example, a date field with user input of 31/12/2018 would be saved in state as 31-12-2018, but the form field can represent it back in the user format again.

Possible solution 1b: A monolithic "autosave" table: Persists across workstations and browsers

What's a use case for this? If your browser hasn't crashed, and you expect to pick up editing from another device or browser, you can just hit save. If your browser, device or system has crashed, you would just reopen that same browser again?

Option 2: Auto-save is a versioning feature

I think it'd be neat to notify authors that other authors have either opened the record they're currently editing, or have made changes since the original author last saved. Neither of these required actually tracking those auto-saved changes in the database though, it's just an event that could be propagated through websockets or even simple HTTP polling.

Or more proactively, we already have the idea of an "edit lock": https://www.silverstripe.org/blog/multi-user-editing-alert/

I think if we went for an actual concurrent editing, we'd have to make server roundtrips fast enough to show to authors before they enter the first keystroke in a potentially conflicting field. The whole form field based CMS UI doesn't lend itself to the free form editing experience of Google Docs, unless you're actually in a rich text field. And there's quite specialised server and client libraries to provide Google Docs like behaviours. We could layer those on top of the actual CMS persistence, I'd be really careful trying to develop that ourselves. 2 min Google brought up https://firepad.io/ :D

With this many throw-away versioned table entries, we'd need a garbage collection task in the background to keep database size in check. That would require adding the requirement of a periodic background job (e.g. through queued jobs) to the baseline CMS installation, which increases complexity of your average LAMP hosting setup. We might need this for other use cases anyway, but just flagging that constraint here.

[clarkepaul]

Heads up that this has come up in conversations when talking about having multiple blocks on a single page which can be expanded/collapsed with fields which aren't saved yet. From a users perspective, having blocks automatically save as you collapse a block or navigate away would be huge step forward. Saving the whole page is still the main action but if autosave existed on content they are editing, people are going to be able to see their creations a lot quicker in the preview.

[chillu]

Auto-saving blocks has the same issues that we described above, just multiplied by even more distinct fields on a page record. The moment we're starting to create two "types" of versions (explicit and implicit saves), it leads to all kinds of follow on issues in the data and interface layer. I don't think this is viable to do as an "implementation detail" of our blocks approach - we should default to page-level block saving, or individual block saving.

I think auto-save is still worth working through as a user experience improvement across the whole CMS, but I wouldn't make our content blocks baseline implementation reliant on it.

[clarkepaul]

I wont quote this user exactly, but you get the idea...

The CMS just deleted all the work I've done today. Thanks. I tried to open a second window to cut and paste text from one page to another. When I check the history tab, it says the last saved draft was yesterday. Its unreliable, worse than Confluence, a difficult feat to achieve.

Its hard to say whether this is a system error, or user having the CMS open in multiple windows and saving the wrong one, or connection issue. But what I think is a valuable takeaway is that its easier to blame the system when the UI is hard to use, auto save can improve this type of issue. At the moment we allow multiple instances (multi-user editing or multi window) which means when auto saving one instance might automatically replace another instance quite easily—something to watch out for UX wise (at the moment we just allow this to happen when saving).

[chillu]

Another way to solve this UX issue in a less impactful way (tech complexity) would be to just alert users when they haven't saved for a while. I understand that sometimes you go hours between saves because you switch back and forth between tabs/systems, I've been there myself. But the CMS could make it more obvious that it's the case. We should also revisit adding multiple editor alerts to more "default installs" in the CMS (e.g. our own project setup, and eventually a recommended addition through silverstripe.org/download).

[maxime-rainville]

Whichever way we go about this, I think we're increasing the risk of collision quite a bit. (e.g. 2 people opening the same page and the CMS starts trying to auto-save things for both of them)

So we need to put some thoughts into how to reconcile those collisions. I guess in a perfect world we would have some sort of Google Doc style multi-user editing where any change is immediately sync to all other users.

[clarkepaul]

Its pretty standard to have content identified as conflicting if its saved within the same time period as other content, then the user must select the right version to go forward—could be an option. There is a module which locks the content for other authors which is another option.

[jonom]

Another way to solve this UX issue in a less impactful way (tech complexity) would be to just alert users when they haven't saved for a while.

Was thinking about this this morning after typing a bunch of content and thinking "Oh shoot, this isn't Apple Notes or Google Docs, I actually have to click Save!" 😄

Found this issue and glad to see that there is thought about auto-saving going on. I would love to see true auto-saving implemented at some point, but in the interim I think a non-intrusive dialogue prompting people to save their work would be very helpful. I'd argue this is becoming urgent, as from my own experience, our ingrained expectations about how work gets saved is changing, and the 'Click to Save' concept is becoming obsolete.

[jonom]

With this many throw-away versioned table entries, we'd need a garbage collection task in the background to keep database size in check. That would require adding the requirement of a periodic background job (e.g. through queued jobs) to the baseline CMS installation, which increases complexity of your average LAMP hosting setup. We might need this for other use cases anyway, but just flagging that constraint here.

Garbage collection could potentially happen on a per-record basis, each time a new version is created or the record is published. Perhaps you keep 2 minute autosaves for the past 30 minutes, then hourly autosaves for the past day, then daily older than that?

[jonom]

Candidate for Experience Debt work?

[clarkepaul]

@jonom Yes it would ideally be included but we are trying to knock off as many non-enhancement type issues prior to opening any larger more complicated pieces of work. The "enhancement" type issues we have included are when there have been regressions in the experience (say from SSv3 - SSv4). I will add it to the Experience Debt board but with our current prioritization of those issues it might not make the top of the list, we'll see.

[maxime-rainville]

I love the enthusiasm, but this looks like something that could take 1-2 devs several weeks of work to implement. I'm not saying it's not worthwhile, but that would make it difficult to do in the context of the experience debt epic.

If we do want to make this priority, then we probably should start by nailing down the general approach we want to take and start subdividing this into smaller tasks.

[chillu]

I think if we implement this, it should be in React forms only. We should not implement this feature twice, once for React, and once for Entwine. Not keen to take on the complexity and ongoing maintenance of that, and then rewrite it all next year. This means it's effectively blocked until page editing forms are managed as React forms.

[unclecheese]

Seems like implementing this on React forms would be painfully simple. Just add middleware to persist the redux-form store to localStorage? I'd be shocked if that didn't already exist.

[jonom]

I'd love auto-saves to be done in the database but if that's too hard then saving to localStorage would be a great start, and maybe enough for most scenarios.

This has been suggested previously - but a stop-gap for now which could address the UX issue of simply forgetting to save - and possibly easy enough to squeeze in to this round of work (🤞) would be to just display some kind of more prominent warning to users that they have unsaved changes. Example:

The Save/Publish buttons already change their state in response to form changes, and I assume this could be bolted on to that existing functionality... so hopefully wouldn't be too difficult? (Technical note - there should probably be a configurable threshold so that a warning isn't displayed until e.g. 2 minutes have gone by since you last saved)

If auto-save does end up being implemented as a client-side-only feature, this warning feature would complement it nicely and remind users to actually commit their changes to the database.

[clarkepaul]

We've introduced the concept of unsaved changes in blocks with a blue dot (in place of DRAFT/MODIFIED) so that could carry through to unsaved changes here (or vice-versa). I like the idea of the warning but not sure on the placement, something we could work with though.

[jonom]

We've introduced the concept of unsaved changes in blocks with a blue dot (in place of DRAFT/MODIFIED) so that could carry through to unsaved changes here (or vice-versa)

I would lean toward vice-versa. I haven't seen the blue dot, and maybe it does work well in practice, but it sounds a bit too subtle to me. I think dots are suitable for indicating draft/modified state because it's just informative, we're not trying to signal a problem or dangerous situation. I think if we're alerting a user to the risk of losing unsaved content we probably need something a bit more attention grabbing.

Maybe these two ideas could complement each other though? In the case of blocks you could put a blue dot on each block that has changed, but also display some sort of warning in the sticky footer pane. Might even be able to extend that to Pages and other DataObjects and mark changed fields with a blue dot.