astronomer / astro-cli

CLI that makes it easy to create, test and deploy Airflow DAGs to Astronomer
https://www.astronomer.io
Other
361 stars 71 forks source link

Add retries for 5xx errors on `astro deploy` #1722

Open neel-astro opened 2 months ago

neel-astro commented 2 months ago

When deploying images to the registry, sometimes there are 5xx errors returned from the registry for various reasons which are mostly intermittent, Astro CLI should look to retry with a small delay on such errors.

Look into whether such retries must be configurable, enabled, or disabled by default when working on the issue.

Also check out if the error could be captured and if we could return a meaningful error instead of exit code 1, in our retry messages

Slack conversation: https://astronomer.slack.com/archives/CL44PA4DB/p1726738192456099

neel-astro commented 1 month ago

TODO: investigate how to capture the error from push command cleanly for both docker & podman engine, and define the errors when we want to retry