Status of AIP-47

[potiuk]

Summary of ALL issues: for 46 issues Completed and closed issues: 44/46: 95.65% Completed files 245/277 = 88.45%

Issues that are not completed and are still opened:

• Migrate Google example DAGs to new design: 71/89 : 79.78%

• Migrate Google system tests to new design: 57/71 : 80.28%

  Issues that are completed and are already closed:

• Amazon AppFlow System Test

• Migrate dbt example DAGs to new design

• Migrate Zendesk example DAGs to new design

• Migrate Yandex example DAGs to new design

• Migrate Trino example DAGs to new design

• Migrate Telegram example DAGs to new design

• Migrate Tableau example DAGs to new design

• Migrate Snowflake example DAGs to new design

• Migrate Slack example DAGs to new design

• Migrate Singularity example DAGs to new design

• Migrate Salesforce example DAGs to new design

• Migrate SQLite example DAGs to new design

• Migrate Qubole example DAGs to new design

• Migrate Presto example DAGs to new design

• Migrate Postgres example DAGs to new design

• Migrate Plexus example DAGs to new design

• Migrate Papermill example DAGs to new design

• Migrate Opsgenie example DAGs to new design

• Migrate Neo4j example DAGs to new design

• Migrate MySQL example DAGs to new design

• Migrate Microsoft example DAGs to new design

• Migrate Jenkins example DAGs to new design

• Migrate JDBC example DAGs to new design

• Migrate Influx example DAGs to new design

• Migrate HTTP example DAGs to new design

• Migrate GitHub example DAGs to new design

• Migrate Elasticsearch example DAGs to new design

• Migrate Docker example DAGs to new design

• Migrate DingTalk example DAGs to new design

• Migrate Databricks example DAGs to new design

• Migrate Cncf.Kubernetes example DAGs to new design

• Migrate Asana example DAGs to new design

• Migrate Apache example DAGs to new design

• Migrate Amazon example DAGs to new design

• Migrate Alibaba example DAGs to new design

• Migrate Airbyte example DAGs to new design

• Migrate Snowflake system tests to new design

• Migrate Postgres system tests to new design

• Migrate Microsoft system tests to new design

• Migrate HTTP system tests to new design

• Migrate CNCF system tests to new design

• Migrate Asana system tests to new design

• Migrate Apache Beam system tests to new design

• Migrate Amazon system tests to new design

[potiuk]

cc: @bhirsz @chethanuk-plutoflume @ferruzzi @Bowrna

[Bowrna]

@potiuk Following is the fix for Migrate Elasticsearch example DAGs to new design PR: https://github.com/apache/airflow/pull/22811

I am currently working on this issue Migrate Docker example DAGs to new design

[potiuk]

@Bowrna - the #22811 missed deletion of the original example. That's why I reopened it :).

[Bowrna]

@Bowrna - the #22811 missed deletion of the original example. That's why I reopened it :).

I have raised PR https://github.com/apache/airflow/pull/24172 to fix this @potiuk Thanks for pointing it to me :)

[potiuk]

@Bowrna - the #22811 missed deletion of the original example. That's why I reopened it :).

I have raised PR #24172 to fix this @potiuk Thanks for pointing it to me :)

Merged already and reflected in status :).

[potiuk]

Updated statis.

[potiuk]

Updated stats:

Summary of ALL issues: for 45 issues Completed and closed issues: 38/45: 84.44% Completed files 147/275 = 53.45%

Issues that are not completed and are still opened:

https://github.com/apache/airflow/issues/22465: 0/1 : 0.00% https://github.com/apache/airflow/issues/22447: 38/89 : 42.70% https://github.com/apache/airflow/issues/22438: 13/36 : 36.11% https://github.com/apache/airflow/issues/22436: 0/1 : 0.00% https://github.com/apache/airflow/issues/22430: 29/71 : 40.85% https://github.com/apache/airflow/issues/22426: 0/9 : 0.00%

[potiuk]

Slowly but surely we are getting there :)

Summary of ALL issues: for 45 issues Completed and closed issues: 39/45: 86.67% Completed files 156/275 = 56.73%

Issues that are not completed and are still opened:

• Migrate Slack example DAGs to new design: 0/1 : 0.00%
• Migrate Google example DAGs to new design: 41/89 : 46.07%
• Migrate Amazon example DAGs to new design: 15/36 : 41.67%
• Migrate Airbyte example DAGs to new design: 0/1 : 0.00%
• Migrate Google system tests to new design: 32/71 : 45.07%
• Migrate Amazon system tests to new design: 0/9 : 0.00%

[Bowrna]

@potiuk Could you check if Slack example DAGs are already merged? I found this PR merged #25137

[potiuk]

Yep. Just re-run the script and closed it.

Summary of ALL issues: for 45 issues Completed and closed issues: 40/45: 88.89% Completed files 171/275 = 62.18%

Issues that are not completed and are still opened:

• Migrate Google example DAGs to new design: 47/89 : 52.81%
• Migrate Amazon example DAGs to new design: 20/36 : 55.56%
• Migrate Airbyte example DAGs to new design: 0/1 : 0.00%
• Migrate Google system tests to new design: 35/71 : 49.30%
• Migrate Amazon system tests to new design: 0/9 : 0.00%

[eladkal]

we seem to miss some? for example: https://github.com/apache/airflow/blob/main/airflow/providers/arangodb/example_dags/example_arangodb.py

[potiuk]

If It was added later then it is likely. Maybe someone can add a review after that as a cleanup.

[ferruzzi]

Yeah, I've noticed the same in the AWS package, some new ones have been added since so we'll have to do a "final pass" after the checklist is cleared.

[potiuk]

Summary of ALL issues: for 45 issues Completed and closed issues: 41/45: 91.11% Completed files 183/275 = 66.55%

Issues that are not completed and are still opened:

• Migrate Google example DAGs to new design: 52/89 : 58.43%
• Migrate Amazon example DAGs to new design: 22/36 : 61.11%
• Migrate Google system tests to new design: 39/71 : 54.93%
• Migrate Amazon system tests to new design: 0/9 : 0.00%

[potiuk]

Summary of ALL issues: for 45 issues Completed and closed issues: 42/45: 93.33% Completed files 209/275 = 76.00%

Issues that are not completed and are still opened:

• Migrate Google example DAGs to new design: 58/89 : 65.17%
• Migrate Amazon example DAGs to new design: 27/36 : 75.00%
• Migrate Google system tests to new design: 45/71 : 63.38%

[potiuk]

Summary of ALL issues: for 45 issues Completed and closed issues: 42/45: 93.33% Completed files 213/275 = 77.45%

Issues that are not completed and are still opened:

• Migrate Google example DAGs to new design: 59/89 : 66.29%
• Migrate Amazon example DAGs to new design: 29/36 : 80.56%
• Migrate Google system tests to new design: 46/71 : 64.79%

[eladkal]

@potiuk i think we can close this one?