feat(core): add global jobs metrics to the server

kestra-io / kestra

Infinitely scalable, event-driven, language-agnostic orchestration and scheduling platform to manage millions of workflows declaratively in code.

https://kestra.io

Apache License 2.0

7.02k stars 408 forks source link

feat(core): add global jobs metrics to the server #4165

Open loicmathieu opened 2 days ago

loicmathieu commented 2 days ago

This PR adds 3 metrics to the worker:

kestra_worker_job_thread: number of threads to process jobs
kestra_worker_job_pending: number of jobs pending to be process (waiting for a thread to be available)
kestra_worker_job_running: number of jobs running / processing (using a thread)

Those metrics will allow to autoscale Worker pods based on the worker job pending and currently running counts. The thread count can be used to compute a usage in percent.

sonarcloud[bot] commented 2 days ago

Quality Gate passed

Issues
3 New issues
0 Accepted issues

Measures
0 Security Hotspots
92.9% Coverage on New Code
0.0% Duplication on New Code

See analysis details on SonarCloud