Some ideas on adopting Flux on server-client syncing

[tiye]

I'm tired working on this idea. So I decide to put it here in case someone need it. So here is my code(CoffeeScript, WebSocket, chatting, not even finished): https://github.com/mvc-works/sync-chat I had an article and a screencast too, but unfortunately they are in Chinese: http://segmentfault.com/blog/jiyinyiyong/1190000001638575 http://www.tudou.com/listplay/tNlbjgmXAJg/EoKUKOXe1eo.html

So here's what I was thinking:

The problem

In single page apps, we have models on both servers and clients. Given that my app would be realtime, there is event pushing too, meaning the same operation on models take place in both the server and the client.

Possible solution from React

We already know how React treact data updates and DOM updates. It's just rerender all virtual DOM(from Store Data), diffs the virtual DOM, and patches on real DOM. How about considering data like that?

Store Data --> Data in the Database virtual DOM --> A structured data generated from Database(say C) real DOM --> each single share of C which represents what client needs to render a whole page

In this case, Database is better to be a piece of data in memory, so it renders fast. Since it's large, maybe we can call some old data as history and save them in a disk database, but keep the recent ones fast.

Data flow I tried

In my code, I divided the code on server into several parts:

• Models: Store(messages, users, threads), States(pagination, caches, configs)
• Controllers: manipulates Models(including states)
• Router: listen to WebSocket and call Controllers
• Scene: structures data that reprecents all the things
• Views: each single share of Scene, belongs to each client
• Sender: send view changes to client(also send direct error messages)

and data flows like this:

It's not show in the graph that Models are splited into Store and States. That makes each client's data like a Component, which rerenders on props changes or state changes. Even further, there might be several views that renders in different frequency(like previewing typing needs frequent one).

What I found

1. Diff/patch JSON is the main problem. I tried jiff(related) and jsondiffpatch. Not as good as expected but worth loooking into.
2. Need framework to do that, the scalfforld code might be quite silly and boring. But rendering just React makes the code quite familiar with patterns we already used in React.

My demo is not finished so I only got these two. Also performance issue is worring, I don't have an good idea.

This idea came from the problems I encountered when I was trying to scale my small single page apps to multiple users. Hope it helps on related problems.

---

updates:

More details can be found at: https://github.com/Cumulo And that chat demo is refactored to be more precise: https://github.com/Cumulo/chat-distract And it's architech is slightly changed:

[pedroteixeira]

It might also be relevant to take a look at JSON OT approach: https://github.com/share/ShareJS

I also considered it, but ended up transfering data via 'JSON Diffs' - just because it was easier to generate then, and apply it server-side using other languages.

[tiye]

OT is interesting and I'd like to look into it.

In my case, diffing takes place on server side, and patching on client, while OT is likely doing the opposite.

[garbles]

Have you considered creating a custom dispatcher that:

1) instead of firing the registered callbacks on a dispatch(), it triggers a websocket event 2) on a websocket event it fires the registered callbacks

[tiye]

@garbles I suppose you are trying to think the server like Store and clients as Component, with websocket as the dispatcher. But in React, components need to access Store really quickly, while requesting Data from server is slow. So I chose to put a copy of data at client-side.

[garbles]

@jiyinyiyong true, though I suppose you could update the DOM immediately and only rollback the change if there is an issue on the server. Seems very complex though.

[tiye]

@garbles It can be quite complecated in some cases, for example, a sinlge user operation that triggers sevaral data changes, it's not as simple as making DOM manipulations and rollback. But I think Flux's uni-data is really good at solving this.

[tiye]

Not practical for large scale apps by now, closing this issue. In case someone might be interested please read https://github.com/Cumulo instead.