dbt-cloud-cli incorrect API token causes unhelpful Python language error

grumBit commented 3 weeks ago

Error description

A TypeError: 'NoneType' object is not subscriptable error is returned when an incorrect API token is passed into dbt-cloud-cli and these lines run;

https://github.com/data-mie/dbt-cloud-cli/blob/cb2680d353727f895b958fd8c5a8fec7657962b6/dbt_cloud/cli.py#L120-L121

Because the response for a failed request does not include a data field, Python gives a TypeError: 'NoneType' object is not subscriptable error, which does not help identify the cause of the problem.

Steps to reproduce the error

To reproduce the error, replace thexxxxxxs below with valid values and run the following;

python -m venv .venv
source .venv/bin/activate
pip install dbt-cloud-cli
DBT_CLOUD_API_TOKEN="BADTOKEN" DBT_CLOUD_ACCOUNT_ID="xxxxxx" DBT_CLOUD_ENV_ID="xxxxxx" DBT_CLOUD_JOB_ID="xxxxxx" dbt-cloud job run --git-branch "xxxxxx" --wait --cause "Bad API token testing"

Result;


Traceback (most recent call last):
  File "/tmp/test/.venv/bin/dbt-cloud", line 8, in <module>
    sys.exit(dbt_cloud())
             ^^^^^^^^^^^
  File "/tmp/test/.venv/lib/python3.11/site-packages/click/core.py", line 1157, in __call__
    return self.main(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/tmp/test/.venv/lib/python3.11/site-packages/click/core.py", line 1078, in main
    rv = self.invoke(ctx)
         ^^^^^^^^^^^^^^^^
  File "/tmp/test/.venv/lib/python3.11/site-packages/click/core.py", line 1688, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
                           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/tmp/test/.venv/lib/python3.11/site-packages/click/core.py", line 1688, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
                           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/tmp/test/.venv/lib/python3.11/site-packages/click/core.py", line 1434, in invoke
    return ctx.invoke(self.callback, **ctx.params)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/tmp/test/.venv/lib/python3.11/site-packages/click/core.py", line 783, in invoke
    return __callback(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/tmp/test/.venv/lib/python3.11/site-packages/dbt_cloud/cli.py", line 123, in run
    run_id = response.json()["data"]["id"]
             ~~~~~~~~~~~~~~~~~~~~~~~^^^^^^
TypeError: 'NoneType' object is not subscriptable

Possible fix

Replace the lines linked above with;

    if wait:
        if not response.ok:
            import json
            raise DbtCloudException(f"Job run failed. Response: {json.dumps(response.json(), indent=2)}")

        run_id = response.json()["data"]["id"]

Which gives a more helpful error;

dbt_cloud.exc.DbtCloudException: Job run failed. Response: {
  "status": {
    "code": 403,
    "is_success": false,
    "user_message": "Access denied: User API Keys are deprecated. Please use account scoped PAT.",
    "developer_message": null
  },
  "data": null
}

grumBit commented 2 weeks ago

Hi @stumelius, would you be the right person to take a look at this? Alternatively, if it would help I'm happy to create a PR with the checks I've seen on others. E.g.;

[ ] I have written unit tests for new features (e.g., a new command test case in conftest.py)
[ ] I have written integration tests for new features (e.g., a new step in the CI workflow)
[ ] I have added descriptions to new commands and arguments
[ ] I have updated the README.md (if applicable)

stumelius commented 2 weeks ago

@grumBit Thanks for bringing this up! I agree with your approach. Would you mind opening a PR for this? I can review it

data-mie / dbt-cloud-cli