In the scheduler container, we host multiple Airflow subprocesses, namely the scheduler, the DAG processor, and the triggerer. We need to generate per-subprocess metrics so we know when/if some processeses start others.
Acceptance Criteria
For each sub-process, generate CPU and memory metrics.
This should be as automated as possible, so any time we create multiple sub-processes in a single container, it is easy or no-effort to generate metrics for all sub-processes.
Overview
In the scheduler container, we host multiple Airflow subprocesses, namely the scheduler, the DAG processor, and the triggerer. We need to generate per-subprocess metrics so we know when/if some processeses start others.
Acceptance Criteria
Additional Info
Context on this: https://github.com/aws/amazon-mwaa-docker-images/pull/59#discussion_r1616436262