search

aws-samples / amazon-mwaa-examples

Amazon Managed Workflows for Apache Airflow (MWAA) Examples repository contains example DAGs, requirements.txt, plugins, and CloudFormation templates focused on Amazon MWAA.
MIT No Attribution
106 stars 61 forks source link

feat: adding mwaa export import dags for version 2.7.2 2.8.1 #69

Closed pshen closed 6 months ago

pshen commented 7 months ago

3 major changes in database schema between 2.6.3 and 2.8.1

Table: task_instance New column custom_operator_name varchar(1000)

Table: dag_run New column clear_number [Integer] not null

Table: slot_pool New column include_deferred[BOOLEAN] NOT NULL

2.6.3 https://airflow.apache.org/docs/apache-airflow/2.6.3/database-erd-ref.html

2.8.1 https://airflow.apache.org/docs/apache-airflow/2.8.1/database-erd-ref.html

Issue #, if available:

Description of changes: Since AWS MWAA has released 2.8.1, this PR is adding 2 dags of export and import for this version.

By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.

pshen commented 7 months ago

Hi @crupakheti,

I've uploaded the dags for version 2.7.2 as well.

I just found I missed one change in table slot_pool in the previous commit. I've fixed it and also enabled to export this table to csv file.

2 major changes in database schema between 2.6.3 and 2.7.2

Table: task_instance New column custom_operator_name varchar(1000)

Table: slot_pool New column include_deferred[BOOLEAN] NOT NULL

2.6.3 https://airflow.apache.org/docs/apache-airflow/2.6.3/database-erd-ref.html

2.7.2 https://airflow.apache.org/docs/apache-airflow/2.7.2/database-erd-ref.html

Ready for your review.

pshen commented 6 months ago

Hi @crupakheti, thanks for your fix. Just merged your PR to the branch.

crupakheti commented 6 months ago

@pshen, perfect! Appreciate your contribution to this project! Thank you!