Export functionality is not working

[ArshikDhar]

Hi Team,

I am using mwaa with airflow version 2.5.1. When the pause state machine gets invoked, it is constantly getting failed in the Export metadata phase with the following error. Looking forward to a resolution

**{ ![Uploading StateMachine Execution - Error.PNG…]()

"dag": "mwaa_export_data", "dag_run": "manual__2024-04-08T18:53:53+00:00", "tasks": [ "export_variable => queued", "export_data => queued", "export_connection => queued", "export_active_dags => running", "back_up_activedags => success", "pause_dags => success", "clean_up => None", "notify_success => None", "activate_dags_on_failure => None" ], "status": "Fail" }**

[crupakheti]

Hey @ArshikDhar, can you please share log of the export_active_dags task?

[ArshikDhar]

Added the error logs from each failed stages + the requirement.txt file too

dag_id=mwaa_export_data_run_id=manual__2024-04-10T14_57_03.964955+00_00_task_id=export_active_dags_attempt=1 (1).log dag_id=mwaa_export_data_run_id=manual__2024-04-10T14_57_03.964955+00_00_task_id=export_active_dags_attempt=1.log dag_id=mwaa_export_data_run_id=manual__2024-04-10T14_57_03.964955+00_00_task_id=export_data_attempt=1.log requirements.txt

[crupakheti]

In the error log, I saw the ModuleNotFoundError: No module named 'smart_open' error. Could you update your requirements.txt file for MWAA to include the smart_open python library and let me know if that resolved the issue?

[crupakheti]

Hold on, I see that you have smart_open in your requirements.txt

[crupakheti]

Are you running your environment in private mode? Does your VPC/subnets have internet connectivity to download the external libs like smart_open?

[ArshikDhar]

Ah that might be the issue. Is there a way I can download it and pass it at the runtime while configuring the airflow?

[crupakheti]

Yes, you can! Please review this doc for details.

[crupakheti]

Closing this issue for now, please reopen it if you need further assistance on this. Thank you!