sync functional, but not near-realtime

[heckj]

After having updated MeetingNotes (https://github.com/automerge/MeetingNotes/pull/42), the sync is clearly functioning, but its no longer happening in realtime.

When I activate websocket channel from two devices to an upstream automerge-repo server, they both appear to sync, but the content isn't replicated realtime.

When I active local peer2peer, the changes replicate faster, but on live updating of text views in MeetingNotes, there are no changes flowing back into the UI - so typing on one doesn't appear on the other.

My tests to date have been iOS <-> Mac:

• just websocket on each
• just peer-to-peer on each
• both websocket and peer-to-peer on each

The consistency of behavior makes me think there's at least some issue in the interface from automerge-repo-swift and what's happening in the docs to replicate changes, and I suspect it's on the "incoming change from remote peer" path.

[heckj]

watching the console/logging while I type on macOS <-websocket-> automerge-repo, as I'm typing in one of the notes fields, there's no immediate replication of content - no sync messages are being generated. I'm a bit unclear if that's in MeetingNotes, AutomergeText, or the repository, but somewhere the "hey, there's a change" signal is getting dropped, delayed significantly (longer than a few seconds), or lost

[heckj]

WebSocket investigation log (Mac <--websocket--> automerge-repo)

From typing a single character in an agenda-item text field:

These first two updates were from the Automerge doc's objectWillChange sink, happening almost exactly 1 second apart. There's a debounce in this stream of 1 second, but getting two update noticifations for a single keypress is a tad unexpected.

3BwAK2fZ3Nrgn9HRBb9yvNx9xXKX ** objectWillChange (1 sec delay) **
Loading model updates
3BwAK2fZ3Nrgn9HRBb9yvNx9xXKX ** objectWillChange (1 sec delay) **
Loading model updates

It goes on to trigger a sync upstream

RESOLVE document id 3BwAK2fZ3Nrgn9HRBb9yvNx9xXKX [ready]
RESOLVED! :: 3BwAK2fZ3Nrgn9HRBb9yvNx9xXKX [ready]
Providing stored sync state for doc 3BwAK2fZ3Nrgn9HRBb9yvNx9xXKX
REPONET: sending message on AutomergeRepo.WebSocketProvider to Optional("storage-server-sync-automerge-org")
REPONET: sending message on AutomergeRepo.PeerToPeerProvider to Optional("storage-server-sync-automerge-org")
Sending SYNC[documentId: 3BwAK2fZ3Nrgn9HRBb9yvNx9xXKX, sender: 16C39119-27E3-4BB9-8905-DDE4DD3CB8FD, target: storage-server-sync-automerge-org, data: 247 bytes] to peer storage-server-sync-automerge-org
Unable to find a connection to peer storage-server-sync-automerge-org
WebSocket received: SYNC[documentId: 3BwAK2fZ3Nrgn9HRBb9yvNx9xXKX, sender: storage-server-sync-automerge-org, target: 16C39119-27E3-4BB9-8905-DDE4DD3CB8FD, data: 71 bytes]
REPONET: received event from network adapter: NetworkAdapterEvents.message[SYNC[documentId: 3BwAK2fZ3Nrgn9HRBb9yvNx9xXKX, sender: storage-server-sync-automerge-org, target: 16C39119-27E3-4BB9-8905-DDE4DD3CB8FD, data: 71 bytes]]
PEER: 16C39119-27E3-4BB9-8905-DDE4DD3CB8FD - handling a sync msg from storage-server-sync-automerge-org to 16C39119-27E3-4BB9-8905-DDE4DD3CB8FD
 - Sync request received for document 3BwAK2fZ3Nrgn9HRBb9yvNx9xXKX
 - working on handle for 3BwAK2fZ3Nrgn9HRBb9yvNx9xXKX, state: ready
Providing stored sync state for doc 3BwAK2fZ3Nrgn9HRBb9yvNx9xXKX

And then we see another objectWillChange from the document, which I thought was also unexpected, but looks like it may have been triggered internally to the repo. I suspect I've got some extra change notifications in the repo doc-resolve process that may be pushed extra notifications:

3BwAK2fZ3Nrgn9HRBb9yvNx9xXKX ** objectWillChange (1 sec delay) **
Loading model updates
Updated contents of document 3BwAK2fZ3Nrgn9HRBb9yvNx9xXKX, state: ready
Storing updated sync state for doc 3BwAK2fZ3Nrgn9HRBb9yvNx9xXKX and peer storage-server-sync-automerge-org.
RESOLVE document id 3BwAK2fZ3Nrgn9HRBb9yvNx9xXKX [ready]
RESOLVED! :: 3BwAK2fZ3Nrgn9HRBb9yvNx9xXKX [ready]
Providing stored sync state for doc 3BwAK2fZ3Nrgn9HRBb9yvNx9xXKX
3BwAK2fZ3Nrgn9HRBb9yvNx9xXKX ** objectWillChange (1 sec delay) **
Loading model updates

[heckj]

Thread on discord highlighted a possible avenue to track discrete sets of patches after applying changes - the diff API. That's not (yet) wired through to the Swift library, so I've opened an issue (https://github.com/automerge/automerge-swift/issues/148) on that repo to wire that up for potential use - including in solving this issue, or related issues in Automerge-swift that aren't efficiently propagating change notifications.

[heckj]

I found at least one place there's a flaw - logged as https://github.com/automerge/automerge-swift/issues/149, AutomergeText doesn't have any direct means to report its changed when an underlying doc changed. In the previous setup in MeetingNotes, I was signaling updates from the coordinator (now repo), and all those places really need to be refined to reflect changes on the Automerge document, or derive a change from that notification.

[heckj]

I have a first cut of this resolved with https://github.com/automerge/automerge-swift/pull/150, but I'll need to cut a full release of automerge-swift to verify that resolves the problems I'm seeing with MeetingNotes, and to establish tests that validate the changes signals propogate as expected.

The implementation was the worst version possible, just to verify that was the issue, and there's pending work to "make that better" with https://github.com/automerge/automerge-swift/issues/151 and https://github.com/automerge/automerge-swift/issues/152

[heckj]

I suspect I have this properly sorted, with updates in Automerge-swift, but to properly test it here along with automerge-repo-swift, I really need to cut a new automerge-swift release. I'll verify the solution, or continue digging, when that's done and I can iterate and test with the fixes.