Consider solutions to avoid losing data when using tiddlyhost on multiple devices

[simonbaird]

Or even multiple tabs.

E.g. https://www.reddit.com/r/TiddlyWiki5/comments/tia329/tiddlyhost_from_multiple_devices/

Ideas:

• Refuse to save if newer changes are detected on the server.
• Some kind of lock or semaphore so only one tab/device has write permission. Show an alert or notice if you don't have write permission. Would need to consider how the lock should time out.
• Try to do a "merge" save, (will be complicated if there are deletes or if the same tiddler is changed in both places).
• Check every minute or so for newer changes and sync them automatically (I think TW has support for that kind of thing when running with the node server, but I'm not sure how it works).

[fmatosqg]

I think those make sense, but I'd also like to align then with my main use cases

• edit on one device (phone while on the street/downstairs) then quickly abandon device and start editing on another (laptop on desk)

I'd say there's enough value on a simple implementation of detecting changes + alert + manual refresh that it could be it's own pr / issue. Potentially rolling the lock and merges as separate issues. I'd love the merge idea btw, I've been using monkey GTD and I guess most of my cases will probably involve writing in laptop and reading/marking done on phone.

[saqimtiaz]

I have the following work on WebDAV:

• Some kind of lock or semaphore so only one tab/device has write permission. Show an alert or notice if you don't have write permission. Lock's don't time out but can be overridden.
• Check every minute or so for newer changes and sync them automatically. Over WebDAV this is pretty fast but pulling down entire wikis every minute over a remote HTTP connection might not be the best choice, maybe just the tiddlers in JSON format?

If I can find the time I might try to extract this into a more generic plugin, currently it relies on WebDAV for things like the modification date.

[simonbaird]

What code is doing the "check every minute"? Something built into TW already, or some custom code?

[simonbaird]

Here's some WIP that might improve the current situation:

• https://github.com/Jermolene/TiddlyWiki5/pull/6589
• https://github.com/simonbaird/tiddlyhost/pull/220

It does depend on switching from the upload saver to the put saver.

@saqimtiaz What do you think about that TW PR?

[saqimtiaz]

What code is doing the "check every minute"? Something built into TW already, or some custom code?

Custom code. TW has a concept of syncers for which you can write adaptors - this is what the node.js version of TW uses. I had some code laying around from something else that did what I needed though so I never explored writing an adaptor for the syncer.

[simonbaird]

220 is deployed now to prod. Here's how you can try the new behavior:

• Create a new site via https://tiddlyhost.com/sites/new and click "Show advanced settings"
• Select the 5.2.3-prerelease, check "Enable put saver" and click "Create"
• Open the new site in two different tabs. Save some changes in one tab, then switch to the other tab and try to save.

[simonbaird]

It's still not a great experience, but perhaps better than overwriting some important changes by accident.

For a next step, I'd like to investigate checking for new updates every minute or so, and either notifying the user and suggesting they hit reload, or syncing the updates automatically if it's possible.

[simonbaird]

Once TW 5.2.3 is released, we might consider enabling the put saver for all 5.2.3 or newer sites by default.

[fmatosqg]

Checked, a few quirks but they're welcome so it makes me double sure nothing is lost. I think they're coming because I always tick the "autosave", and ticking it triggers a save even though I guess nothing is saved, but still assuming it will generate a new ETAG. I can't reliably reproduce this, but I feel it's related to opening the tab for 1st time, but won't happen after reload.

Worked nice on both (2 tabs on laptop) and (2 tabs on laptop + 1 on phone).

• I'd be keen to give it a run in my main tiddly, is there a way to upstream it to an existing tiddly? Or would it happen naturally once you upgrade the server?
• I'd love to see the reload button going a different color once the code knows there's a conflict, nudging the user to reload

[AnthonyMuscio]

I have only come to this Issue today.

But I have a working proof on concept, but It will take time to finesse for publishing;

• On first use of a tiddlyhost site, in a fresh browser or device you are prompted to provide details which are saved in cookies specifically a device name.
• On open a site as the wiki loads it tests to see if or which device/browser has the wiki "checked out"
  • If locked to another device/browser enter a modal where you are encourage to reload to see if its now available
  • The ability to save is turned off or obscured
  • Display which browser device and contact info (usually oneself) currently owns the wiki.
  • A read only access is available (see also move to local storage)
  • If not locked by and browser/device take ownership and save back to the wiki, it is now owned by this browser/device
  • I plan to make this possible for a minimal period eg 15 seconds since last load, otherwise you need to reload again and check it is not now locked out.
  • If locked by this Browser device no modal is presented just the wiki with autosave on
  • When finished there is a custom save and checkin to release the wiki for another browser/device and return it to read only

The above could be enhanced by including the username in the browser/device/username cookies (optional password Obscured but stored cookie and/or in wiki) but of course by definition most users will be using a different browser/device.

I would like to set a timeout on inactivity that automatically checked the wiki back in and released it. However with a single user, its easy to release from one device when finished. Or say when moving from mobile to desktop or return you most likely have both available.

• You could overwrite the check out if necessary
• We may still be able to later save changes made to a wiki that was not checked out (see move to local storage)

One advantage of tiddlyhost wikis is they exist at a url and unlike file wikis the cookies can be made unique to that url.

• To reduce the chance of two open tabs opening the same browser/device, since they have the same browser/device, I use window.name and target to ensure the link to this wiki always opens in the same tab.

Move to local storage

• There may be a way to allow changes on a copy of the wiki, that does not have it checked out, have the changes moved to local storage, and once that browser device gets to check out the wiki then providing a tool to apply the changes, which most often will not be in-contention and provide a diff tool for those in contention.
• This may also help if one returned to a browser/device for which you forcefully overrode the checkout eg: work desktop was not checked back in before you left work.
• If we have multiple users the local storage could be managed under a user namespace to smooth the later merging.

[pmario]

I think there is no need for the server to be involved. There is the https://developer.mozilla.org/en-US/docs/Web/API/Broadcast_Channel_API which allows us to create a message that is broadcasted, whenever a new tab, window or even an iframe is opened from the same orgin.

If a tab is opened with an eg: ID=wiki-1 it broadcasts this info. .. Every other tab which is already open and contains a wiki did already create a "broadcast listener". If it gets an info that a new tab is opened with it's own ID it, sends a message back, that this wiki ID is already open. -- The newly created window / tab can now display a message, that this wiki is already open.

I'm using a very similar mechanism with my file-backups plugin. Since it's a browser addON the mechanism is slightly different, but the principle is the same. It works very well for me.

This mechanism works with the same browser only. So if 2 different device are involved, that's not possible.

[simonbaird]

The solution I'm imagining involves changes to TiddlyWiki to do the following:

• Every minute or so, do an OPTIONS request similar to the one done on startup to detect the dav header and enable the put saver.
• If the etag in the response header has changed, alert the user in some way, perhaps with a popup message "New changes detected. Please reload to ensure you don't lose thos changes!".
• There could be also a visual indicator to show that saving the site now would be destructive. The actual UX can be tweaked from there, e.g. if we want to entire disable the save button at that point it could be done.

I think the amount of code needed would be small, since TiddlyWiki already has functions to do the OPTIONS request and store the etag. So it's probably a small-ish patch to compare old and new values and trigger the alert.

It should work across different devices, e.g. phone vs laptop, and it feels a lot simpler than inventing some kind of write permission lock workflow.

[simonbaird]

Re "is there a way to upstream it to an existing tiddly", if you enable put saver (in the Tiddlyhost settings for your site) it should work, but you won't get the longer explanation text when the save is refused unless your TiddlyWiki is a 5.2.3-prerelease.

Actually in 5.2.2 it says:

Error while saving

File changed on server

Versus this on the 5.2.3 and higher:

Error while saving

File changed on server

The site has been updated since you first loaded it. Saving now would cause those                                                                                                 
updates to be overwritten.

Try reloading and then reapplying your changes.

[simonbaird]

FYI the put saver mentioned above is now used by default for new sites.

I think this issue is a little stale now. Todo: Extract out the viable sugestions from above and file them as separate issues.

[pmario]

I think this issue is a little stale now. Todo: Extract out the viable sugestions from above and file them as separate issues.

I think that's a good idea.

[fmatosqg]

I'm curious, how do I upgrade my existing site or clone it in a way that it supports this feature?

On Wed, 7 Dec 2022, 21:51 Mario Pietsch, @.***> wrote:

I think this issue is a little stale now. Todo: Extract out the viable sugestions from above and file them as separate issues.

I think that's a good idea.

— Reply to this email directly, view it on GitHub https://github.com/simonbaird/tiddlyhost/issues/219#issuecomment-1340773648, or unsubscribe https://github.com/notifications/unsubscribe-auth/ABI4Y4OKWVUTUSR6ZRS4MCDWMBT3LANCNFSM5SNLMDMQ . You are receiving this because you commented.Message ID: @.***>

[simonbaird]

I'm curious, how do I upgrade my existing site or clone it in a way that it supports this feature?

See https://github.com/simonbaird/tiddlyhost/wiki/About-the-two-different-save-methods .