[server] Switch to bi-directional communication

[joshuaauerbachwatson]

Although the current app is "mostly client server" it has a push-based aspect because the latest game state (and player lists in the early phases) need to be multicast for efficient operation. In particular, while using http it is hard to eliminate the polling, which is just wasting cycles and bandwidth. Also, using explicit http POST operations, with my semi-manual JSON serialization and separate declarations in Swift and Go, is a pain in the neck.

There are "official" gRPC implementations for Go and for Objective-C on IOS and a slightly less official set of Swift bindings for the latter. My current reading of gRPC documentation suggests that I can use bi-directional channels and mixture of synchronous RPC, asynchronous RPC, and simple protobuf sends to get exactly the communication semantics I want. The declarative language for rpcs and messages can become the single point of definition for the protocol, with bindings generated for both languages.

The one thing I'm not sure about is the extent to which App Platform would get in my way. Right now, the server is doing simple http and is relying on App Platform to manage the external URL with simple binding of paths ... presumably, it is converting https to http in the process. To use the provided gRPC channels out of the box I believe I would need SSL / TLS end-to-end and and would add a custom bearer token protocol on top of that. I might have to craft my own gRPC custom channel type to get through the impedence mismatches. This will be a combination of experimentation and chatting up people at DO to get answers.

[joshuaauerbachwatson]

If it turns out that App platform does get in the way, and I don't want to engineer around the problem with software, I could also just run a VM (droplet) with the server on it and use standard gRPC channels protected by SSL/TLS. The lowest cost droplet plan is actually cheaper than the cheapest App Platform plan. Of course, there is a bunch of extra maintenance work involved which is why AP is attractive if it works.

[joshuaauerbachwatson]

When I originally opened this issue I was convinced that gRPC was the solution but a subsequent discussion with Davi DeBarros at DigitalOcean convinced me that the quickest path to an adequate bi-directional solution was to use websockets. This would likely cover the most important "pain point" by eliminating the polling. True that I would not get the benefit of stub generation for both Swift and Go, but the protocol is currently simple enough that I don't really need that.

[joshuaauerbachwatson]

A tentative plan.

1. Use gorilla at the server side (a go implementation of websockets).
2. Initially, just deploy a second server which is pretty much exactly the gorilla sample chat application.
3. Initially, just add a chat channel to the basic anyCards app which uses the second server.

At this point we have gotten our feet wet with websockets. The chat channel is probably useful anyway. Going forward from this point,

• add a schema to the websocket path so that it can handle game state as well as chat and has the minimal authentication based on the app token
• retire the old server and standardize on the new one

I have so far gotten the new server deployed on app platform. Some minor changes to the sample program were needed so that the existing HTML access could continue to work. These were just changes to the websocket URL (wss instead of ws and also using the /websocket route to distinguish this server from the old one).

I tried and rejected writing a new test client using SwiftUI. The idea was I could steal some more sample code here and there and get the basic functionality working. And, getting my feet wet with SwiftUI seemed like a good idea too, so maybe we could convert the entire app to use it. But, too much learning curve. I can get the basic websocket thing going more easily by being more incremental

[joshuaauerbachwatson]

The chat chennel is now working.

[joshuaauerbachwatson]

Before trying to merge the chat and game-state channels and standardize on web sockets, I need to get bearer token authentication working with web sockets. I have digressed a bit into the auth0 integration question but I see that there is a lot of work, potentially, in that item. So, I may initially use the old burned-in app token rather than tokens provided by auth0.

[joshuaauerbachwatson]

It is now clear that web sockets work fine through App Platform and I could start using it to do the push notifications (and perhaps integrated chat). However, there seems to be a problem in that web sockets don't integrate bearer token authentication very well. Perhaps this doesn't matter a lot since I can hand-roll this as part of the data structures that are passed in (as before), and just rely on disconnecting those users who fail authentication. I think there are two designs to consider and I'm not sure which is better.

1. Limited use of web sockets. This would involve having two end points, one which is traditional https and the other which upgrades to web socket. All the stuff that fits into client/server goes to the traditional endpoint, which uses auth0 infrastructure in the recommended way. Requests that open push or chat channels go to the web socket endpoint but perhaps only after the user has been validated through the traditional endpoint and has been given a one-time secret. This must be passed when opening the web socket.
2. Figure out how to make the web socket access fully secure and integrated into auth0, after which everything can be built on it.

Note that some decision about how to integrate a bearer token into web socket is common to both.

[joshuaauerbachwatson]

Note that some decision about how to integrate a bearer token into web socket is common to both.

On the Swift side, the URLSessionConfiguration can specify headers. The trick is that they don't want you to specify the authorization header. It seems, however, that people do specify it successfully. To avoid any later problems, if you control both client and server (as I do), then you can get around any long-term risk by just using an alternate (non-standard) header instead of authorization. That sounds like a plan.

On the go side, since upgrade takes an http request as its argument, you can parse out the header from it and subject that to whatever validation is required.

Despite this being relatively straightforward, I am inclined to use websocket only for chat and for push and to use internally generated tokens for validation. In the short run, the rest of the protocol can use the app token as it does at present.

It would be good to keep the planned auth0 integration orthogonal and only switch over to it when we are ready to ditch the app token in favor of something generated by auth0.

[joshuaauerbachwatson]

I am inclined to use websocket only for chat and for push and to use internally generated tokens for validation. In the short run, the rest of the protocol can use the app token as it does at present.

Observations.

1. There is a difference between using two endpoints and actually having two servers. Right now I have two servers for expedience in quickly exploiting the Gorilla sample code, but I was getting charged twice as much monthly by Digital Ocean as a consequence, so I shut down the second server to save money while I was not really working on AnyCards. My plan now is to use the Gorilla sample only as education and a possible cut-and-paste source but to integrate all new functionality into the existing server. The go SDKs should easily handle this.
2. Once a player has joined a game, he must have the websocket channel for game state pushes and should be free to start using it for chat. So, the entire first-connection sequence should be integrated into one module. With this approach it is easy to generate a one-time random ID token for each user.

It would be good to keep the planned auth0 integration orthogonal and only switch over to it when we are ready to ditch the app token in favor of something generated by auth0.

Since I am already part way down the path with auth0 and have learned a little more about how to complete the authentication handshakes on the server side, I think it will be more efficient to work on the two changes together. I have therefore closed issue #13.

[joshuaauerbachwatson]

The schema for what is sent on the bi-directional channel can be nearly trivial.

• Every transmission is just a type token followed by either a chat contribution or a game state.
• In the upload direction, the only valid type token is 'chat' (otherwise, we would need to complicate the protocol by adding responses; the idea is to use the old http based channel for all request/response flows).
• A chat contribution is just a string. It takes the form<User>: <Message> where the user is the id of the originator. Chat contributions can flow either way.
• A game state only flows in the download direction and it consists of an encoded game state, I won't worry now about whether the game state schema is promoted to be part of this schema or whether it starts out as an opaque string and goes through a second decode.

I would expect the schema on the request/response channel to remain largely the same, in the short run, other than

• removing the poll/response sequence
• retiring the app token from the individual messages, in favor of using auth0 for all security

At present, I plan to retain "Nearby Only" mode because, if anything, the two communication styles are now better aligned semantically than they were before. If I need to ditch it, I will. If I keep it, I should consider some logic to sort out multiple ongoing games in the same "proximity." RIght now, the protocol assumes that everyone who is running AnyCards in proximity is in the same game.

[joshuaauerbachwatson]

Implementation sequence.

• [ ] Complete the auth0 integration on the existing https channel and drop the app token.
• [ ] Create the second endpoint and figure out how it gets authenticated and integrated into the auth0 handshake. Don't have it do anything yet.
• [ ] Institute the new protocol on the bi-directional channel and start using it for chat.
• [ ] Switch over the "new game state" logic to use the new channel. Try to align the two communication types better.
• [ ] FIgure out what to do about chat in Nearby mode. Perhaps it is disabled?