search

anvilproject / client-apis

Clients for Python, R, javascript that interact with [terra, gen3, galaxy, others]
Apache License 2.0
9 stars 5 forks source link

Retrieved terra access token unable to authenticate to https://staging.theanvil.io (but works on https://staging.datastage.io) #22

Closed bwalsh closed 3 years ago

bwalsh commented 4 years ago

After setting client up per documentation, client unable to authenticate with endpoint. The data flow for the client is documented here

 python3 -m pytest --user_email compbio.ohsu@gmail.com  --log-level DEBUG  --gen3_endpoint https://staging.theanvil.io  tests/integration/test_auth.py

>           raise HTTPError(http_error_msg, response=self)
E           requests.exceptions.HTTPError: 401 Client Error: UNAUTHORIZED for url: https://staging.theanvil.io/api/v0/submission/

../../.virtualenvs/pyAnVIL/lib/python3.7/site-packages/requests/models.py:940: HTTPError
---------------------------------------------------------------- Captured log call ----------------------------------------------------------------
test_auth.py                13 DEBUG    attempting retrieval of graphql schema
connectionpool.py          959 DEBUG    Starting new HTTPS connection (1): staging.theanvil.io:443
connectionpool.py          437 DEBUG    https://staging.theanvil.io:443 "GET /api/v0/submission/getschema HTTP/1.1" 200 None
test_auth.py                15 DEBUG    OK retrieval of graphql schema
test_auth.py                17 DEBUG    attempting retrieval of programs list
gen3_auth.py                55 DEBUG    __call__, https://staging.theanvil.io/api/v0/submission/ adding Authorization header
gen3_auth.py               106 DEBUG    get gcloud_access_token ['gcloud', 'auth', 'print-access-token', 'compbio.ohsu@gmail.com']
gen3_auth.py               111 DEBUG    gcloud_access_token ya29....<TOKEN HERE >
connectionpool.py          959 DEBUG    Starting new HTTPS connection (1): broad-bond-dev.appspot.com:443
connectionpool.py          437 DEBUG    https://broad-bond-dev.appspot.com:443 "GET /api/link/v1/fence/accesstoken/ HTTP/1.1" 200 997
gen3_auth.py               124 DEBUG    Terra access token expires in 8:19:42.563095
connectionpool.py          959 DEBUG    Starting new HTTPS connection (1): staging.theanvil.io:443
connectionpool.py          437 DEBUG    https://staging.theanvil.io:443 "GET /api/v0/submission/ HTTP/1.1" 401 105
gen3_auth.py                82 DEBUG    _handle_401, cleared _access_token, retrying with new token
gen3_auth.py               106 DEBUG    get gcloud_access_token ['gcloud', 'auth', 'print-access-token', 'compbio.ohsu@gmail.com']
gen3_auth.py               111 DEBUG    gcloud_access_token ya29.<TOKEN HERE>
connectionpool.py          959 DEBUG    Starting new HTTPS connection (1): broad-bond-dev.appspot.com:443
connectionpool.py          437 DEBUG    https://broad-bond-dev.appspot.com:443 "GET /api/link/v1/fence/accesstoken/ HTTP/1.1" 200 997
gen3_auth.py               124 DEBUG    Terra access token expires in 8:19:42.783052
connectionpool.py          437 DEBUG    https://staging.theanvil.io:443 "GET /api/v0/submission/ HTTP/1.1" 401 105
======================================================= 1 failed, 1 passed in 26.68 seconds =======================================================

Terra Account setup:

image

Gen3 Account setup:

image

Note: the client works against the https://staging.datastage.io endpoint

---------------------------------------------------------------- Captured log call ----------------------------------------------------------------
test_auth.py                13 DEBUG    attempting retrieval of graphql schema
connectionpool.py          959 DEBUG    Starting new HTTPS connection (1): staging.datastage.io:443
connectionpool.py          437 DEBUG    https://staging.datastage.io:443 "GET /api/v0/submission/getschema HTTP/1.1" 200 None
test_auth.py                15 DEBUG    OK retrieval of graphql schema
test_auth.py                17 DEBUG    attempting retrieval of programs list
gen3_auth.py                55 DEBUG    __call__, https://staging.datastage.io/api/v0/submission/ adding Authorization header
gen3_auth.py               106 DEBUG    get gcloud_access_token ['gcloud', 'auth', 'print-access-token', 'compbio.ohsu@gmail.com']
gen3_auth.py               111 DEBUG    gcloud_access_token ya29.<TOKEN HERE>
connectionpool.py          959 DEBUG    Starting new HTTPS connection (1): broad-bond-dev.appspot.com:443
connectionpool.py          437 DEBUG    https://broad-bond-dev.appspot.com:443 "GET /api/link/v1/fence/accesstoken/ HTTP/1.1" 200 997
gen3_auth.py               124 DEBUG    Terra access token expires in 8:19:42.608317
connectionpool.py          959 DEBUG    Starting new HTTPS connection (1): staging.datastage.io:443
connectionpool.py          437 DEBUG    https://staging.datastage.io:443 "GET /api/v0/submission/ HTTP/1.1" 200 None
test_auth.py                20 DEBUG    {'links': ['/v0/submission/topmed', '/v0/submission/parent']}
test_auth.py                22 DEBUG    attempting retrieval of project list
gen3_auth.py                55 DEBUG    __call__, https://staging.datastage.io/api/v0/submission/topmed adding Authorization header
connectionpool.py          959 DEBUG    Starting new HTTPS connection (1): staging.datastage.io:443
connectionpool.py          437 DEBUG    https://staging.datastage.io:443 "GET /api/v0/submission/topmed HTTP/1.1" 200 None
gen3_auth.py                55 DEBUG    __call__, https://staging.datastage.io/api/v0/submission/parent adding Authorization header
connectionpool.py          959 DEBUG    Starting new HTTPS connection (1): staging.datastage.io:443
connectionpool.py          437 DEBUG    https://staging.datastage.io:443 "GET /api/v0/submission/parent HTTP/1.1" 200 None
test_auth.py                28 DEBUG    OK: Authenticated from https://broad-bond-dev.appspot.com/api/link/v1/fence/accesstoken/ to https://staging.datastage.io projects: {'links': ['/v0/submission/parent/CARDIA_IRB', '/v0/submission/parent/CARDIA_IRB-NPU', '/v0/submission/parent/HVH_DS-CVD-IRB-MDS_', '/v0/submission/parent/HVH_HMB-IRB-MDS_', '/v0/submission/parent/CFS_DS-HLBS-IRB-NPU_', '/v0/submission/parent/FHS_HMB-IRB-MDS_', '/v0/submission/parent/FHS_HMB-IRB-NPU-MDS_', '/v0/submission/parent/JHS_DS-FDO-IRB_', '/v0/submission/parent/JHS_DS-FDO-IRB-NPU_', '/v0/submission/parent/JHS_HMB-IRB_', '/v0/submission/parent/JHS_HMB-IRB-NPU_', '/v0/submission/parent/ARIC_HMB-IRB_', '/v0/submission/parent/WHI_HMB-IRB_', '/v0/submission/parent/WHI_HMB-IRB-NPU_']}

Gen3 Account setup:

image

bwalsh commented 4 years ago

status

Retried test with suggested access token endpoints. Decoded the resulting tokens. Unsurprisingly, the iss field for the endpoint that works matches the gen3_endpoint.

works

cat ~/.fissconfig
[DEFAULT]
root_url=https://firecloud-orchestration.dsde-dev.broadinstitute.org/
debug=True

python3 -m pytest --user_email compbio.ohsu@gmail.com --log-level DEBUG --gen3_endpoint https://staging.datastage.io --terra_auth_url https://broad-bond-dev.appspot.com/api/link/v1/fence/accesstoken tests/integration/test_auth.py

{
  "pur": "access",
  "aud": [
    "openid",
    "google_credentials",
    "4EmZnWKVMoPyhdJMh7EB8SSl3Uojo20QfsAR77gu"
  ],
  "sub": "250",
  "iss": "https://staging.datastage.io/user",
  "iat": 1579736107,
  "exp": 1579737307,
  "jti": "4496456a-cd15-4370-aa35-f498308f5f27",
  "context": {
    "user": {
      "name": "WALSBR",
      "is_admin": false,
      "google": {
        "proxy_group": null
      },
      "projects": {}
    }
  },
  "azp": "4EmZnWKVMoPyhdJMh7EB8SSl3Uojo20QfsAR77gu"
}

! work

python3 -m pytest --user_email compbio.ohsu@gmail.com --log-level DEBUG --gen3_endpoint https://staging.theanvil.io --terra_auth_url https://broad-bond-dev.appspot.com/api/link/v1/fence/accesstoken tests/integration/test_auth.py

{
  "pur": "access",
  "aud": [
    "openid",
    "google_credentials",
    "4EmZnWKVMoPyhdJMh7EB8SSl3Uojo20QfsAR77gu"
  ],
  "sub": "250",
  "iss": "https://staging.datastage.io/user",
  "iat": 1579736326,
  "exp": 1579737526,
  "jti": "8569decf-5975-4d47-9ae5-0e74372dc333",
  "context": {
    "user": {
      "name": "WALSBR",
      "is_admin": false,
      "google": {
        "proxy_group": null
      },
      "projects": {}
    }
  },
  "azp": "4EmZnWKVMoPyhdJMh7EB8SSl3Uojo20QfsAR77gu"
}

python3 -m pytest --user_email compbio.ohsu@gmail.com --log-level DEBUG --gen3_endpoint https://staging.theanvil.io --terra_auth_url https://broad-bond-dev.appspot.com/api/link/v1/dcf-fence/accesstoken tests/integration/test_auth.py
{
  "pur": "access",
  "aud": [
    "openid",
    "google_credentials",
    "OnUikYtSzOK7cKYQgLjWLOzclR4MPbSuVAVPAroK"
  ],
  "sub": "5804",
  "iss": "https://nci-crdc-staging.datacommons.io/user",
  "iat": 1579736489,
  "exp": 1579737689,
  "jti": "103f4d63-2ca8-460c-b18b-2b97962e07de",
  "context": {
    "user": {
      "name": "WALSBR",
      "is_admin": false,
      "google": {
        "proxy_group": null
      },
      "projects": {}
    }
  },
  "azp": "OnUikYtSzOK7cKYQgLjWLOzclR4MPbSuVAVPAroK"
}