Resumable DDP subscriptions

[raix]

Its not possible to resume a DDP connection - I'm mainly thinking about pub/sub.

Why?

• Less data on the wire
• Better support for offline data caching
• Perhaps less in memory subscription tracking on the server

Brainstorm:

The server could have a different way to keep track of subscriptions - perhaps the client is better fittet requesting the data not on the client:

The client could tell the server what subscriptions are on the client - this way the server could query the db directly using the publish query on the client, the server time stamp and the resumed publish query.

I'll prop. write a prototype of this since its part of getting ground:db to work better with subscriptions and the iron:router.

• [ ] Add last data server time stamp to all data (could also be used in conflict resolution)
• [ ] Make sub/pub handle data more intelligent using the servertime stamp
• [ ] Make sub/pub intelligent about before / after arguments in the publish

This is not ideal to solve in packages since it requires replacing things in ddp/livedata etc. but thats been the message from @glasser so far:

Regarding server timestamp on ddp handshake: I don't think this belongs as a core part of DDP. You can always add methods (or subscriptions) that return server time. And unlike this PR, which hooks directly into the low level DDP code, that could easily be an Atmosphere package.

Note: The initial code indicating resumable subscriptions were swiped from Meteor approx: a year ago (or something - I havent tracked down a reference, but its part of a clean up by glasser)

Note about resume:

• https://github.com/meteor/meteor/blame/v0.5.6/packages/livedata/livedata_server.js#L688-L698

Ref server resume:

• https://app.asana.com/0/159908330244/577350817064

Ref about server timestamp:

• https://github.com/meteor/meteor/pull/1865#issuecomment-38090181
• https://github.com/meteor/meteor/pull/1527
• ground:connection - caching ddp - TEST
• ground:subscriptions - WIP

[mitar]

It there a message from @glasser missing in the description above?

[raix]

just added references, and removed the label "MDG dont agree" since its not 100% clear

[mquandalle]

This could also be used for server side rendering. For instance the server send the 10 first items of a list in the initial HTTP response. Then the client could resume this DDP connection so we don't have to send the same data twice (is that still the case in fast-render @arunoda?).

[arunoda]

We don't have any kind of resuming in fast-render but it does do conflict resolving upto some point. If we need to do resumable stuff I think we need to manage changes in a way how git works.

So, that's a big topic.

[mitar]

If we need to do resumable stuff I think we need to manage changes in a way how git works.

What has git with this?