search

sassoftware / sas-airflow-provider

Apache Airflow Provider for creating tasks in Airflow to execute SAS Studio Flows and Jobs.
Apache License 2.0
18 stars 15 forks source link

DAG fails with "The folder could not be found.","path: /studioDevelopment/code" after successful authentication to Viya environment #31

Closed pb01123 closed 9 months ago

pb01123 commented 9 months ago

Viya version: Stable 2024.01 Release: 20240127.1706321865048 Airflow version: 2.8.1

Here's the log SAS-airflow-DAG_error_09022024.txt

The flow is created and saved at a location (which is dags_folder in airflow.cfg) using SAS Studio.

Error snippet: [2024-02-09T02:04:57.580-0500] {sas.py:82} INFO - Get oauth token (see README if this crashes) [2024-02-09T02:04:58.013-0500] {logging_mixin.py:188} INFO - A new unnamed compute session will be created [2024-02-09T02:05:17.564-0500] {logging_mixin.py:188} INFO - Compute session 9e39aa3d-1cfa-4d54-a1d7-5bfc8cb75d67-ses0000 created [2024-02-09T02:05:17.564-0500] {sas_studio.py:183} INFO - Generate code for Studio object: /data/viyauser0/flows/simple-flow1.flw [2024-02-09T02:05:17.564-0500] {sas_studio.py:296} INFO - Code generation for Studio object stored in Content [2024-02-09T02:05:17.724-0500] {taskinstance.py:2698} ERROR - Task failed with exception Traceback (most recent call last): File "/airflow-venv/lib64/python3.11/site-packages/sas_airflow_provider/operators/sas_studio.py", line 184, in execute res = self._generate_object_code() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/airflow-venv/lib64/python3.11/site-packages/sas_airflow_provider/operators/sas_studio.py", line 312, in _generate_object_code raise RuntimeError(f"Code generation failed: {response.text}") RuntimeError: Code generation failed: {"version":2,"httpStatusCode":500,"errorCode":124303,"message":"Error encountered during code generation.","details":["cause: The folder could not be found.","path: /studioDevelopment/code","correlator: a97ad249-13d7-4037-a06a-8116dda914c3"],"errors":[{"version":2,"httpStatusCode":404,"errorCode":5121,"message":"The folder could not be found."}]} During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/airflow-venv/lib64/python3.11/site-packages/airflow/models/taskinstance.py", line 433, in _execute_task result = execute_callable(context=context, **execute_callable_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/airflow-venv/lib64/python3.11/site-packages/sas_airflow_provider/operators/sas_studio.py", line 208, in execute raise AirflowException(f"SASStudioOperator error: {str(e)}") airflow.exceptions.AirflowException: SASStudioOperator error: Code generation failed: {"version":2,"httpStatusCode":500,"errorCode":124303,"message":"Error encountered during code generation.","details":["cause: The folder could not be found.","path: /studioDevelopment/code","correlator: a97ad249-13d7-4037-a06a-8116dda914c3"],"errors":[{"version":2,"httpStatusCode":404,"errorCode":5121,"message":"The folder could not be found."}]} [2024-02-09T02:05:17.728-0500] {taskinstance.py:1138} INFO - Marking task as FAILED. dag_id=simple-flow1, task_id=simple-flow1.flw, execution_date=20240209T070454, start_date=20240209T070457, end_date=20240209T070517 [2024-02-09T02:05:17.728-0500] {sas_studio.py:249} INFO - Deleting session with id 9e39aa3d-1cfa-4d54-a1d7-5bfc8cb75d67-ses0000 [2024-02-09T02:05:18.104-0500] {sas_studio.py:252} INFO - Compute session succesfully deleted

pb01123 commented 9 months ago

Well, it turns out that the path= in SASStudioOperator task needs to be a path in content and I had saved the .flw on file system. Once I changed that, the dag run was a success.

Reference: https://developer.sas.com/apis/rest/DataManagement/#sas-studio-development-codegeneration

Closing...