RFC: Patching subscriptions together

[mattkrick]

Basic subscriptions is all but complete, but patching those subscriptions together is a whole different story.

For example, let's say you're having a meeting. That meeting has many Participants and each participant has many socket Connections (eg different tabs, different devices, etc).

There are 2 approaches:

create a subscription on the table join Subscribing to a join, if possible, keeps things easy, but makes for a huge payload because every time a connection changes, the entire participant object would get sent down the wire. This could be mitigated by caching everything sent to a particular socket channel on the server. Then, only send down the diff in a "patch". Instructions on how to assemble that patch are required because, for example, we don't know if [{id:123}] should replace the current value, or append to the current value.

create n+1 subscriptions

We can do this on the server (more complicated)

When a subscription for the participants is created, we start streaming participant documents down the wire. Before each one goes down the wire, we start a subscription for connections that have the given participantId. Then, we wrap the unsubscribe listener with a HOF that also unsubscribes to the new subscription we just started so when we unsubscribe from the parent, we unsub from the children as well. Patching instructions are required in payload just like above.

We can do it on the client (more latency) When a subscription for the participants is created, we start streaming participant documents down the wire. When each doc is received on the client, we subscribe to all connections for each participant. (n+1 round trips, but done in parallel, only about 2n latency). This gives us a new unsubscribe function for each, which we'd probably stick in an array & flush it when we want to unsub. Patching instructions are not required, since you would do the patching in the model-view layer, outside of Cashay, outside of your subscriber. This is the approach that is available today.

Backends vary, and whatever approach the application developer chooses is fine, as long as they send that patch info.

The patch instructions would look like a falcor path. For example when a participant adds a new connection: {payload, type: "ADD", patch: "participants['123'].connections"}. This tells Cashay a few things:

• participants is an array
• we need to find the document in the participants array where id == '123'
• we are adding the new document to the connections array inside that document
• connections is an array. It CANNOT be a point (single-doc) subscription. If you wanted to do that (and I don't know why) you'd just have to access index 0.

UPDATE and REMOVE would need a patch like participants['123'].connections['456'] because mutating an entire array would be invalid.

CC @jordanh

[jordanh]

I think this is just excellent. This morning over coffee I thought through a few of the domain state cases we'll need to implement and I think this covers them all. I'm looking forward to be able to test this in action (pun intended).

[mattkrick]

Take the path Team.foo[123].bar

• This path has 4 "pieces": Team, foo, [123], bar. A piece, per netflix, can be an identifier or indexer, the latter having brackets around it.
• We know Team is an object because foo is an identifier
• When we look at foo, we know it is an array since [123] is an indexer.
• When we look at [123] we know it is an object because bar is an identifier
• When we look at bar, we don't know what it is. 😨

To determine that, we need to use the GraphQL schema (thank goodness we have it on the client!) We just take the ensureNonNull(subSchema.type) If it's an object,

• if ADD, return an object
• if UPDATE, return an object
• if REMOVE, delete from parent (meaning we can't recurse to level n, rather n -1)

if it's an array,

• if ADD, return an array with the new doc appended to the end
• if UPDATE, splice out the old, splice in the new and return the copied array
• if REMOVE, return the array with the old spliced out

this should clean up that really, really ugly code that is currently in use

[mattkrick]

updated in v0.16.0