How would you implement something like phoenix channels

[tcoopman]

So, playing with this for a bit, I was wondering how you would implement something like phoenix channels (https://hexdocs.pm/phoenix/channels.html). Where you can have lots of users joining a channel and communicate with each other.

I guess we'd need some global state that keeps all subscribers? I was wondering if you have any idea how you would implement this?

[yawaramin]

Hi Thomas, I think for something like that we would need:

• Concurrent state management in each server
• A concurrent distributed state update system. I believe Phoenix uses a custom implementation of the Raft protocol but to make things easier we could use a shared Redis server

So yeah definitely possible but not easy by any stretch.

[tcoopman]

What about an easier first step, like making it work for just one server, so that it doesn't need the concurrent state management. Then it would mostly be a place where you keep track of all the subscribers. Would you implement that by passing a global ChannelManager as a first argument into the service, something like this:

| (`GET, ["ws"]) => getWS(channelManager)

Where the channelManager keeps track of the subscribed topics...

Or you would you tackle this in an other way?

[yawaramin]

Ah, assuming a single ReWeb server and no distributed state, WebSocket handlers are still isolated from each other, so if they wanted to communicate with each other they would still need a concurrent message-passing mechanism. Maybe an API like:

module type Cache = sig
  type elem
  type t

  val get : name:string -> t -> elem Lwt.t
  val put : name:string -> elem -> t -> unit Lwt.t
end

Then we could create a cache of this type Cache.t at the module level and pass it in to each WS handler as it gets created. The key would be to ensure concurrent access to the cache.

[yawaramin]

I've implemented a concurrent cache now. Will try to add a 'chatroom' demo to the fullstack-reason repo soon.

[yawaramin]

And while doing that realized I need a more 'channel'-like abstraction. Working on a design for ReWeb.Topic now. Basically an in-memory pub-sub thing.

[tcoopman]

Nice, definitely looking forward to seeing that. If you want any feedback let me know

[yawaramin]

Implemented ReWeb.Topic: https://yawaramin.github.io/re-web/re-web/ReWeb/Topic/Make/index.html

Next step is to demo it in the fullstack-reason repo. I'm planning to build a very simple 'chat app' that you can open on multiple tabs and show that they're all broadcasting to each other. Of course this would not work with multiple instances since they wouldn't be talking to each other–but ReWeb topics can be hooked up to a shared Redis/etc. (theoretically at least).

[tcoopman]

I've played a bit with this already, not sure I'm doing it correctly, so looking forward to your example.

One thing I was wondering if the Topic maybe also needs a publish method that publishes to all except the current subscriber. publishFrom(subscription, message) similar to https://hexdocs.pm/phoenix/Phoenix.Channel.html#broadcast_from/3?

[yawaramin]

Thanks for trying it out! I'm planning to write a rather complete example of a 'chat app' in the fullstack-reason repo.

Publishing to all except the current subscriber would be a super-simple addition, I think it's a great idea. Signature would be val publish_from : 'a subscription -> msg:'a -> unit Lwt.t.

Do you want to take a crack at it this weekend? Else I'll probably add it next week.

[tcoopman]

Unfortunately I probably won't have any time in the next week, but if you haven't found the time yet, I'll take a stab at it

[yawaramin]

Done: https://github.com/yawaramin/re-web/commit/aa6f6cb52da0658a5ea2a37bf0bf95ee04f17f99

[yawaramin]

Implemented a simple chat server in https://github.com/yawaramin/fullstack-reason/commit/1cb9346dcdf72ec1b87970e22e5fff6dc87117c3

[tcoopman]

Thanks for this. I've got one question about the implementation (not sure if you get notified of comments on code - so asking here): https://github.com/yawaramin/fullstack-reason/commit/1cb9346dcdf72ec1b87970e22e5fff6dc87117c3#r37226409