search

joelkuiper / patavi

A distributed system for wrapping R as a web service
patavi.com
Other
3 stars 9 forks source link

Do not require a thread for each (concurrent) WAMP RPC #9

Open joelkuiper opened 10 years ago

joelkuiper commented 10 years ago

Right now there are 256 (upped from the 4 default from http-kit) threads doing nothing (i.e. waiting for the workers to finish) because clj-wamp requires thread-blocking when resolving an RPC call. This is probably due to the assumption that the RPCs are relatively short-lived and happen on the same machine; this is not true in our case since they're long lived and distributed across several workers.

There should be a way to separate request and response for WAMP RPC calls so a single thread can be used which only waits and keeps the connections open. Likely this means a different WAMP implementation.

joelkuiper commented 10 years ago

@cgmartin maybe you have an idea about this, or can I go ahead and fork your stuff?

cgmartin commented 10 years ago

Hi @joelkuiper please feel free to fork, I'd be interested in seeing what you come up with. I can understand the problem of the thread-blocking during the RPC call and worker distribution, but I don't have a clear idea for a solution at the moment.