StorminStanley / st2workroom

Vagrant environment used to play with StackStorm, develop StackStorm for your environment, or develop on StackStorm itself!
Apache License 2.0
23 stars 21 forks source link

[WIP] [gunicorn] Use 1 "sync" worker type with 1 process and 1 thread for st2api #350

Closed Kami closed 8 years ago

Kami commented 8 years ago

It turns out that "eventlet" worker type doesn't work well with the code which relies on eventlet monkey patching and performs a lot of I/O. Performance and throughput is very bad (it simply looks like things run in sync more or less).

After a lot of digging in and checking things out this change seems to fix that. Now it's basically very similar to local dev setup - we have a single (gunicorn worker) process which runs stuff in async because of the eventlet monkey patching.

TODO:

Kami commented 8 years ago

Sadly it turns out this won't out so well with stream endpoint so we need to figure something out.

One option would be to run another gunicorn worker and app just for stream endpoint. That gunicorn app would use async worker while others (api, auth, potentially mistral, but I haven't investigated Mistral yet) would use a sync one. This does add some overhead (additional "app"), but, imo it's not such a bad idea (I was already thinking about it in the past) since stream and other API endpoints have fundamentally different models and requirements (long lived vs short lived requests) so separating those requests is usually the way to go.

For backward compatibility reasons, we could do some nginx proxying "magic" and still expose it under the same path.