teamhephy / controller

Hephy Workflow Controller (API)
https://teamhephy.com
MIT License
14 stars 26 forks source link

feat(controller): permit setting GUNICORN_WORKERS #137

Closed kingdonb closed 3 years ago

kingdonb commented 3 years ago

permit setting GUNICORN_WORKERS

This is the chart-only change. #75 had another separate change for CONN_MAX_AGE, but I could not get controller to build tonight... this variable is already mentioned in controller v2.21.2, though!

This chart-only change makes GUNICORN_WORKERS settable from in the controller chart.

75 will close and we can reopen with whatever changes are needed for CONN_MAX_AGE, if that's still something we want to configure. (I think we weren't convinced this other setting is really needed, whenever that conversation last left off...)

kingdonb commented 3 years ago

I tested this chart change, and it works. Setting gunicorn_workers to 5 and exec'ing into controller, I can see 6 python processes running (5 workers). Leaving the chart value unset, I can see 18 processes running for python (17 workers, my 4 core machine * 4 + 1, according to the comment in values.yaml, this is exactly as expected)

5 is probably a sensible default value for my clusters, but it would not be a sensible default for multitenant Workflow providers. 🤷‍♂️ I think this is the right change, it's easy to accidentally make your nodes too big, and saturate database with too many connections just based on this calculation.

YMMV. Serious operators should probably examine this value and determine if it is sensible for themselves. Now, they can at least be able to override it when the value is wrong.

kingdonb commented 3 years ago

This got tested on a Rancher RKE quickstart cluster at 1.17.14 tonight, and it performed as expected.

dmcnaught commented 3 years ago

Moved this into two of our clusters for testing