Merging subtasks

[mrzapp]

My wife and I are using this app for shopping lists and other similar operations. When she and I are checking off subtasks simultaneously, we overwrite each other's changes when CalDAV syncing, as they are stored as plain text.

I want to help implementing some merge logic, where should I start in the code?

[mrzapp]

Correction: Subtasks are called "checklists" in the app

[dmfs]

Hmm... that's not so much a feature for OpenTasks. What you describe is called a "sync conflict", i.e. a value was modified on both synced sources. It's the sync app which should detect this case and act accordingly. The easy solution is to let one of both sources win automatically (i.e. "server always wins" or "client always wins"), but as you stated that's not always the desired option

Merging two different check lists is actually quite a challenge because you need to know what exactly was changed on both ends. That means in addition to the new state on the local and the remote end you also need to know the last common state to derive the changes on both ends and merge them.

Another option would be to store the diff of every modification and then replay the changes after the sync app marks the task as "synced" to get the new version (sort of what git rebase does).

It is certainly possible to implement this in OpenTasks alone without support in the sync app, but it wouldn't be optimal. We would have to make check lists a task provider feature (at presents it's only a UI thing)

Here is how it could work:

• whenever a task is modified locally we create a shadow copy of the task (or optionally just the check list)
• subsequent local changes won't create more shadow copies, only the first one is kept
• when a sync app modified the task, we look up the original check list and derive the changes applied by the sync app
• when we know the changes which have been made remotely, we can
  • apply them to the latest version of the check list
  • remove the shadow copy
  • mark the task dirty
  • trigger another sync, so the client can pick up the changes and send them to the remote end

There could still be sync conflicts, like when one item is modified on one source and deleted on the other source. We need to solve them by adding additional constraints and rules, like check list items are unique (you can't have two items with the same text).

We'd also make sure that we still support smart sync apps which are able to apply diffs on their own. This could by done by allowing a sync app to delete the shadow copy to signal the task provider that it must not rebase the changes.

Also, this assumes the sync app uses a "server wins" conflict resolution strategy. If it uses a "client wins" strategy this won't work at all, because the sync app will discard any remote changes and OpenTasks will never see them.

[mrzapp]

I agree, ideally I think this should be a standard handled by the sync app. However, we find ourselves in a quite different situation, where we can only have checklists stored as text descriptions, and parsing is up to the client. We also have no influence on the sync app and its features.

I have a slightly different suggestion to the logic, inspired by yours:

• Whenever the sync client updates the local task:
  • Make a shadow copy of the checklist, if none exists
  • Check if any local changes were made since the last sync by comparing the current local checklist to the shadow copy
  • If changes were made:
  • Do a diff not of the remote version vs. the local version, but of each version vs. the shadow copy
  • Commit the changes from each version to the shadow copy:
    • Add any items that don't exist on the shadow copy, but do exist on either the local or the remote version
    • Remove any items that do exist on the shadow copy, but don't exist on either the local or remote version
    • Remove duplicate items
  • Trigger a new sync
  • If no changes were made:
  • Overwrite the local version with the remote version

The first notable change from your logic is that the shadow copy is not created upon change, but upon sync

The second notable change is that it's not "remote always wins" vs. "local always wins", it's more of a "previous remote copy (shadow copy) is the last reference point, and new remote and local versions are change descriptions"

Does that make sense?