Allow multiple WAMP servers to be run concurrently.

[jasongilman]

I'm working on integration testing a combination Clojure/ClojureScript application that uses clj-wamp. The server side would be started within normal test environment and the ClojureScript side would run inside a phantomjs environment. I wanted some way to control actions within the phantomjs environment and I thought using a separate WAMP server to communicate with the running phantomjs instance would work. I don't think the current server implementation will allow multiple instances since it defines some top level refs in vars. My whole reason for wanting multiple concurrent servers might be a really bad idea but it still points to way the library could be improved.

Stuart Sierra had a recent blog post and a talk at Clojure West called Clojure in the Large that talked about some better approaches for handling state in applications and libraries. The short version is that libraries should avoid putting things in vars and instead just return a context object that holds the state and pass that back into library functions. The clj-wamp server code could probably be refactored pretty easily to put the refs inside a map or record and returned as a "server" context object. Functions that send events etc. would take this context object as an argument. The normal use case for clj-wamp could be to just stick the server object in a var and pass it back in. It would still be easy for users of the library but allow more flexibility in how it's used.

[cgmartin]

Good stuff. I like the idea of encapsulating state using a returned context. I'll check out that Clojure West talk.

As long as all of the client wamp session-ids are unique during add-client, multiple wamp servers should be able to run concurrently. The ref maps are all keyed by this wamp session-id. There's an enhancement in the release-1.1 branch that allows you to generate your own custom session id as well, ie. where you could salt them with an instance number.

I really appreciate the feedback and links. I'm still wrapping my head around functional programming and Clojure best practices. Thanks again @jasongilman .

[cgmartin]

Actually, I may not have understood your original post... Is the 2nd wamp server (for communicating to PhantomJS) a separate process altogether or is running from the same process context as the first server? If a separate process, then you're right, you'd run into problems. If multiple httpkit servers are run in the same process it has a better chance of working, but you might still run into problems. :) I'm not sure now if it would have issues broadcasting PubSub events across multiple httpkit servers/channels in the same process.

For sharing events across separate server processes, ie. a cluster of servers, a separate pubsub queuing system like Redis comes to mind... Where each wamp server manages her own set of WebSocket clients, and incoming WS messages are pubsubbed to/from Redis by each of the wamp servers and rebroadcasted out. Sometime soon I mean to try this out.

I digress, but to your original note about controlling PhantomJS clients... Instead of a separate wamp server, could you control it with a separate wamp client that communicates with the same/single server? Like sending an event to a pubsub channel that the PhantomJS client could listen/subscribe to? Wanted to share that thought in case it works for your setup - If not, I'd be curious your use case for controlling PhantomJS via the server.

[jasongilman]

I'm building another library on top of clj-wamp that will allow capturing and sending data for visualizations to a web browser. I wanted to integration test it using phantomjs. I want to be able to assert things like the client has the data. My idea was to use a separate cli-wamp server to open a separate web socket just for communicating test instructions to the browser. This would all be in process. I'm 70% convinced its actually a bad idea and overly complicated for testing so I don't think I'll actually use it that way.

I still think changing clj-wamp to be less reliant on vars is a good idea though. It's going to lead to a generally more flexible design to stuff that state in a context object that gets passed around.

On Wed, Aug 21, 2013 at 11:44 PM, Christopher Martin < notifications@github.com> wrote:

Actually, I may not have understood your original post... Is the 2nd wamp server (for communicating to PhantomJS) a separate process altogether or is running from the same process context as the first server? If a separate process, then you're right, you'd run into problems. If multiple httpkit servers are run in the same process it has a better chance of working, but you might still run into problems. :) I'm not sure now if it would have issues broadcasting PubSub events across multiple httpkit servers/channels in the same process.

For sharing events across separate server processes, ie. a cluster of servers, a separate pubsub queuing system like Redis comes to mind... Where each wamp server manages her own set of WebSocket clients, and incoming WS messages are pubsubbed to/from Redis by each of the wamp servers and rebroadcasted out. Sometime soon I mean to try this out.

I digress, but to your original note about controlling PhantomJS clients... Instead of a separate wamp server, could you control it with a separate wamp client that communicates with the same/single server? Like sending an event to a pubsub channel that the PhantomJS client could listen/subscribe to? Wanted to share that thought in case it works for your setup - If not, I'd be curious your use case for controlling PhantomJS via the server.

— Reply to this email directly or view it on GitHubhttps://github.com/cgmartin/clj-wamp/issues/10#issuecomment-23066660 .

[cgmartin]

Ah, OK. Yep, I was curious if there was a workaround in the meantime. I'm still interested in exploring the refactor you mention. Added it to the 1.2.0 list to review once I can get the in-progress 1.1.0 version tested and released. Cheers