search

NASA-PDS / nucleus

Nucleus is a software platform used to create workflows for the Planetary Data (PDS).
https://nasa-pds.github.io/nucleus
Apache License 2.0
0 stars 0 forks source link

Evaluate options to backup and restore the metadata database and logs of MWAA #36

Closed ramesh-maddegoda closed 1 year ago

ramesh-maddegoda commented 1 year ago

The Amazon Managed Workflows for Apache Airflow (MWAA) has some ongoing costs due to the always up and running Airflow UI. There is no option available to suspend the MWAA environment when it is not in use.

This task is to evaluate options to backup and restore the metadata database and logs of MWAA to reduce the operating cost of Nucleus.

jordanpadams commented 1 year ago

Status: Still in work to see if this is an option. Not much information found thus far.

ramesh-maddegoda commented 1 year ago

The following guide discusses about migrating a self-managed Apache Airflow workflows to Amazon MWAA, or upgrade an existing Amazon MWAA environment to a new Apache Airflow version. https://docs.aws.amazon.com/mwaa/latest/migrationguide/about-mwaa-migration.html

ramesh-maddegoda commented 1 year ago

This approach is not a very practical approach in production compared to the other Airflow deployment approaches such as Airflow on EKS or ECS.