I've been wondering if you had a way to help evaluate which stages were the bottlenecks. Something like if the queue fills up in a stage, report that in some performance tuning mode. Perhaps more workers need to be allocated to that stage (or fewer if there is too much context switching), more CPU should be allocated to the container, more RAM, etc. Currently it requires some manual intervention to attempt to ascertain where the bottlenecks are.
Hoy @gtadamson ! This sounds like a great idea. Since version 4.x internally there exists a Supervisor class that has the sole purpose of monitoring workers which makes this feature is more plausible.
I've been wondering if you had a way to help evaluate which stages were the bottlenecks. Something like if the queue fills up in a stage, report that in some performance tuning mode. Perhaps more workers need to be allocated to that stage (or fewer if there is too much context switching), more CPU should be allocated to the container, more RAM, etc. Currently it requires some manual intervention to attempt to ascertain where the bottlenecks are.