search

salemove / freddy

Messaging api supporting request-response and acknowledgements.
MIT License
9 stars 0 forks source link

Fix unbounded parallelism #14

Closed urmastalimaa closed 9 years ago

urmastalimaa commented 9 years ago

Creating new threads forces all thread-local state to be reinitialized. This means creating new PG connections and AR pools and so on. Moreover, some requests timing out can lead to an explosion of parallelism which creates very many threads and as a result of this also very many file handlers for amqp pipes and user connections (e.g PG).

Therefore it is necessary to bound parallelism while still allowing unlimited reqres inside a responder.

This can be done by listening for RPC responses on a dedicated thread and moving consumer threads into a thread pool.

stenver commented 9 years ago

Lgtm

indrekj commented 9 years ago

while we are already fixing concurreny issues, then can you also look into the shared channel issue?

urmastalimaa commented 9 years ago

I know what the problem is (sharing channels between actual jruby threads). I'm not sure how to solve it atm. Would do in another PR.