sassoftware / sas-airflow-provider

Apache Airflow Provider for creating tasks in Airflow to execute SAS Studio Flows and Jobs.
Apache License 2.0
18 stars 15 forks source link

Connection always failed to connect to Viya4 (with AzureAD) from Airflow #34

Open em47390 opened 6 months ago

em47390 commented 6 months ago

Hi experts,

May I know what's the steps to obtain the correct token and permission to access Viya (e.g., create compute session and then run code) from Airflow? The Viya is using Azure AD to login, while Airflow webserver use default (admin/admin). I have tried many ways to create the Viya access_token , but when I start a DAG in airflow, there are always errors returned like this:

This is the defined connection: image

Below are current steps I tried to create access_token, and the error messages:

  1. Sample 1: [ERROR MSG] __File "/home/airflow/.local/lib/python3.8/site-packages/sas_airflow_provider/util/util.py", line 186, in create_or_connect_tosession raise RuntimeError(f"Failed to create session: {response.text}") RuntimeError: Failed to create session: {"version":2,"httpStatusCode":500,"errorCode":30081,"message":"Invalid user: \"scim5.idp\"","details":["traceId: 077fd31fc37606ae","path: /launcher/processes","path: /compute/contexts/4d13c061-10a8-4419-8e5c-bf9017d97d97/sessions","correlator: e8c3e59a-1048-4ea7-a4a0-e96ea1ea3b25"]}

[Steps to get access_token] BEARER_TOKEN=curl -sk -X POST "${INGRESS_URL}/SASLogon/oauth/clients/consul?callback=false&serviceId=scim5.idp" \ -H "X-Consul-Token: $CONSUL_TOKEN"| awk -F: '{print $2}'|awk -F\" '{print $2}' echo "The registration access-token is: " ${BEARER_TOKEN}

curl -k -X POST "${INGRESS_URL}/SASLogon/oauth/clients" \ -H "Content-Type: application/json" \ -H "Authorization: Bearer $BEARER_TOKEN" \ -d '{ "client_id": "scim5.idp", "client_secret": "idpsecret", "authorities": ["SCIM"], "authorized_grant_types": ["client_credentials"], "access_token_validity": 473040000 }'

ACCESS_TOKEN=curl -skX POST "${INGRESS_URL}/SASLogon/oauth/token" \ -H "Content-Type: application/x-www-form-urlencoded" \ -d "grant_type=client_credentials" \ -u "scim5.idp:idpsecret"| awk -F: '{print $2}'|awk -F\" '{print $2}'; \ echo "The client access-token is: " ${ACCESS_TOKEN};

  1. Sample2: [ERROR MSG] _[2024-05-07, 13:51:14 CST] {taskinstance.py:1937} ERROR - Task failed with exception Traceback (most recent call last): File "/home/airflow/.local/lib/python3.8/site-packages/sas_airflow_provider/operators/sas_studio.py", line 164, in execute compute_session = create_or_connect_to_session(self.connection, File "/home/airflow/.local/lib/python3.8/site-packages/sas_airflow_provider/util/util.py", line 166, in create_or_connect_to_session raise RuntimeError(f"Find context named {context_name} failed: {response.status_code}") RuntimeError: Find context named SAS Studio compute context failed: 403 During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/airflow/.local/lib/python3.8/site-packages/sas_airflow_provider/operators/sasstudio.py", line 200, in execute raise AirflowException(f"SASStudioOperator error: {str(e)}") airflow.exceptions.AirflowException: SASStudioOperator error: Find context named SAS Studio compute context failed: 403

[Steps to get access_token] BEARER_TOKEN=curl -sk -X POST "${INGRESS_URL}/SASLogon/oauth/clients/consul?callback=false&serviceId=scim4.idp" \ -H "X-Consul-Token: $CONSUL_TOKEN"| awk -F: '{print $2}'|awk -F\" '{print $2}' echo "The registration access-token is: " ${BEARER_TOKEN}

curl -k -X POST "${INGRESS_URL}/SASLogon/oauth/clients" \ -H "Content-Type: application/json" \ -H "Authorization: Bearer $BEARER_TOKEN" \ -d '{ "client_id": "scim4.idp", "client_secret": "idpsecret", "scope": ["openid"], "authorized_grant_types": ["authorization_code","client_credentials","refresh_token"], "redirect_uri": "urn:ietf:wg:oauth:2.0:oob", "access_token_validity": 473040000, "refresh_token_validity": 473040000 }'

authorization_code: https://xxx.xx.xx.com/SASLogon/oauth/authorize?client_id=scim4.idp&response_type=code

ACCESS_TOKEN=curl -k -X POST ${INGRESS_URL}/SASLogon/oauth/token \ -H "Accept: application/json" \ -H "Content-Type: application/x-www-form-urlencoded" \ -d "grant_type=authorization_code&code=${authorization_code}" \ -u 'scim4.idp:idpsecret' echo "The client access-token is: " ${ACCESS_TOKEN};

  1. Sample3: [ERROR MSG] RuntimeError: Failed to create session: {"version":2,"httpStatusCode":500,"errorCode":30175,"message":"Unable to generate a new OAuth token for current user","details":["traceId: 91de15f83df9aa89","path: /launcher/processes","path: /compute/contexts/4d13c061-10a8-4419-8e5c-bf9017d97d97/sessions","correlator: 8005f2e2-37b3-4e8b-b772-1fc59884c456"]}

    [Steps to get access_token] BEAREER_TOKEN=curl -sk -X POST "https://${INGRESS_URL}/SASLogon/oauth/token" \ -u "sas.cli:" \ -H "Content-Type: application/x-www-form-urlencoded" \ -d "grant_type=password&username=sasboot&password=lnxsas" echo "The registration access-token is: " ${BEARER_TOKEN}

    curl -k -X POST "${INGRESS_URL}/SASLogon/oauth/clients" \ -H "Content-Type: application/json" \ -H "Authorization: Bearer $BEARER_TOKEN" \ -d '{ "client_id": "scim7.idp", "client_secret": "idpsecret", "scope": ["openid"], "authorized_grant_types": ["authorization_code"], "redirect_uri": "urn:ietf:wg:oauth:2.0:oob", "access_token_validity": 473040000, "refresh_token_validity": 31622400 }'

    authorization_code: https://xxx.xx.xx.com/SASLogon/oauth/authorize?client_id=scim7.idp&response_type=code

    ID_TOKEN=curl -k -X POST ${INGRESS_URL}/SASLogon/oauth/token \ -H "Accept: application/json" \ -H "Content-Type: application/x-www-form-urlencoded" \ -d "grant_type=authorization_code&code=dWKOg2_ZVDTX_KDEN_O-eGBpxmbnpIDx" \ -u 'scim7.idp:idpsecret' echo "The client access-token is: " ${ID_TOKEN};