Most of our auto-scaling service metrics were not being emitted due to the direct use of Airflow's StatsD object in a context where the necessary Airflow configurations are not yet set (the entrypoint.py, i.e. the top level container process.) Hence, metrics were simply being ignored. To solve this issue, I made a change to directly create a StatsD object with the right configuration.
Testing:
Confirmed metrics are being emitted:
Notice that the second graph is empty because there were no running tasks (also, this graph was working anyway, so not related to the issue mentioned in this PR.)
By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.
Issue #, if available: N/A
Description of changes:
Most of our auto-scaling service metrics were not being emitted due to the direct use of Airflow's StatsD object in a context where the necessary Airflow configurations are not yet set (the entrypoint.py, i.e. the top level container process.) Hence, metrics were simply being ignored. To solve this issue, I made a change to directly create a StatsD object with the right configuration.
Testing:
Confirmed metrics are being emitted:
Notice that the second graph is empty because there were no running tasks (also, this graph was working anyway, so not related to the issue mentioned in this PR.)
By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.