Connection loss, reconnects (and CAP theorem)

[mariohock]

Hi, I'm thinking about realizing a shopping list app with Flexx. It should support live syncing between the "shoppers" (i.e., users) when there is Internet connection. But it should continue to work if there is no signal in the store.

Currently, as soon as the web socket connection is broken, the Flexx page is cleared and only shows "Lost connection with server". This means I can no longer see my shopping list at all, which would be kind of inconvenient. It seems reasonable simple to disable this behaviour in app/_clientcore.py. But what happens when the connection is back? Is it possible (reasonable to implement a way) to establish a new web socket connection?

Finally the state in the browser (JS part) needs to be synced with the server (Python part) again. E.g., tell the server that I checked-off some items in the meantime. Due to the CAP theorem (https://en.wikipedia.org/wiki/CAP_theorem) it is never trivial to solve this, which means there can't just be a simple built-in solution, since there is always a trade-off involved. This means the app developer would need some freedom to re-establish a consistent state.

A simpler solution in my case would be not to re-sync the state but at least give the JS part the possibility sent its state back to server (i.e., the Python part), let it process the data and then reload the page.

The CAP theorem states that you can never have "consistency", "availability", and "partition tolerance" altogether. You always have to pick one or two. Flexx apparently chose the property "consistency" and sacrifices "availability" and "partition tolerance". In my use case "availability" and "partition tolerance" are crucial while "consistency" could be sacrificed. Is there some interest in the community to give the app developers some more freedom to choose between C, A, and P or is Flexx just the wrong system for this use case?

[Konubinix]

I love the way you put it.

I don't know about CAP theorem, but following from the wikipedia article you shared, it says.

"...states that it is impossible for a distributed data store to..."

And following the link to "distributed data store", I can read.

"Distributed databases are usually non-relational databases that enable a quick access to data over a large number of nodes."

I assume that the CAP theorem might be true in the case of two actors. But might be like this could be a specific use case where the CAP theorem does not actually apply.

Anyway, I also have a use case where I would like to avoid the annoying "Lost connection with server".

A flexx application is currently running in a tablet on my living room, showing recent photograph we took directly from our NAS. When the application is started, I need to click manually on the fullscreen button because web browser don't allow to go fullscreen without an explicit user action (likely for security reasons or whatever).

We stop the wifi at night. On the morning, we have the "Lost connection with server" page. I need to manually refresh the page. This is not a big deal per se, but it is a big deal for me.

I planned to change this behavior in a ad-hoc fashion by adding a bit of javascript in the "Lost connection with server" page to refresh it at 7am, when the wifi is back on. That would not get over the need to manually click of the fullscreen button.

Therefore, I would love to see a way to choose what to do when the connection is lost, while waiting to having it back. In my example, just showing the last seen image (freezing the state in a sense) would be enough I guess.

I understand this is not an easy feature, as you already exposed, and for that reason I honestly don't think it will be part of flexx. But I wanted to share with you that you are not alone with such a dream :-).

My best, -- Konubinix GPG Key : 7439106A Fingerprint: 5993 BE7A DA65 E2D9 06CE 5C36 75D2 3CED 7439 106A

[mariohock]

Thanks for your input, I notice that "partition tolerance" is also relevant for others. :-)

The CAP theorem is proven for "distributed data stores", however, I think it's safe to say that it applys to networking in general. But of course, in some cases the trade-offs are so straight forward that one would not need a theorem to get it. But, as you say, the CAP theorem makes it clear, that the desired behavior in a partition scenario is quite app specific. This means, the app developers need some tools to deal with it.

In your case, I think, there is no potential for conflicting state. This means it should be quite easy to get the state of your app up-to-date, if flexx had the ability to recover from connection loss, i.e., could establish a new websocket connection.

Just for demonstration: Your case would be much harder if there were two tablets and if users could make active inputs (like change the background color or something). If different background colors were chosen on the two tablets during the night, flexx could not resolve this conflicting state automatically. But you could write some app logic to solve it that suits your use case (e.g., just pick one since it's not a critical feature anyway).

I imagine something like that the JS part and the python part keep running during connection loss but any cross-boundary action throws a "catchable" exception. If I catch it, I have to deal with consistency/availability. If I don't catch it we could still display "connection lost". Also both sides should be notified if connection is lost/re-established.

Is it technically possible to just open a new websocket connection when the initial one is broken? Or is there too much going on in the background that would break in this case?