Investigate CE deployment daily run failures

astronomer / astro-sdk

Astro SDK allows rapid and clean development of {Extract, Load, Transform} workflows using Python and SQL, powered by Apache Airflow.

https://astro-sdk-python.rtfd.io/

Apache License 2.0

329 stars 40 forks source link

Investigate CE deployment daily run failures #2090

Closed vatsrahul1001 closed 6 months ago

vatsrahul1001 commented 6 months ago

More context

vatsrahul1001 commented 6 months ago

As tasks were getting killed with return code -9, I investigated the resources of our Astro-SDK deployment and found that it was using an A5 machine (1 CPU, 2GB). Since we were running many DAGs at the same time, tasks were receiving SIGKILL. Additionally, all of our other deployments use A10 machines (2 CPUs, 4GB). I have updated our Astro-SDK deployment to use an A10 machine, and I no longer see that issue.