Error: Unable to login to DBT Cloud

[stumelius]

Hi and thanks for this cool project! I'm unable to authenticate with either DBT_CLOUD_ACCOUNT_ID and DBT_CLOUD_TOKEN env vars or explicitly configuring the provider as follows:

# main.tf
terraform {
  required_providers {
    dbt = {
      source  = "GtheSheep/dbt-cloud"
      version = "0.0.67"
    }
  }
}

provider "dbt" {
  account_id  = <account_id>
  token          = "<token>"
}

data "dbt_cloud_project" "dbt_cloud_project" {
  project_id = var.dbt_cloud_project_id
}

I'm getting a Error: Unable to login to DBT Cloud when running terraform apply. I've verified that the account id and token work with the dbt Cloud API v2:

curl --request GET \
  --url https://cloud.getdbt.com/api/v2/accounts/$DBT_CLOUD_ACCOUNT_ID/projects/ \
  --header "Content-Type: application/json" \
  --header "Authorization: Token $DBT_CLOUD_TOKEN" \
  | jq .

Debug logs

2021-10-27T12:49:05.649+0300 [DEBUG] created provider logger: level=debug
2021-10-27T12:49:05.649+0300 [INFO]  provider: configuring client automatic mTLS
2021-10-27T12:49:05.735+0300 [DEBUG] provider: starting plugin: path=.terraform/providers/registry.terraform.io/gthesheep/dbt-cloud/0.0.67/linux_amd64/terraform-provider-dbt-cloud_v0.0.67 args=[.terraform/providers/registry.terraform.io/gthesheep/dbt-cloud/0.0.67/linux_amd64/terraform-provider-dbt-cloud_v0.0.67]
2021-10-27T12:49:05.736+0300 [DEBUG] provider: plugin started: path=.terraform/providers/registry.terraform.io/gthesheep/dbt-cloud/0.0.67/linux_amd64/terraform-provider-dbt-cloud_v0.0.67 pid=22052
2021-10-27T12:49:05.736+0300 [DEBUG] provider: waiting for RPC address: path=.terraform/providers/registry.terraform.io/gthesheep/dbt-cloud/0.0.67/linux_amd64/terraform-provider-dbt-cloud_v0.0.67
2021-10-27T12:49:05.746+0300 [INFO]  provider.terraform-provider-dbt-cloud_v0.0.67: configuring server automatic mTLS: timestamp=2021-10-27T12:49:05.746+0300
2021-10-27T12:49:05.787+0300 [DEBUG] provider: using plugin: version=5
2021-10-27T12:49:05.787+0300 [DEBUG] provider.terraform-provider-dbt-cloud_v0.0.67: plugin address: address=/tmp/plugin595756387 network=unix timestamp=2021-10-27T12:49:05.787+0300
2021-10-27T12:49:15.899+0300 [INFO]  backend/local: plan operation completed
╷
│ Error: Unable to login to DBT Cloud
│ 
│   with provider["registry.terraform.io/gthesheep/dbt-cloud"],
│   on main.tf line 10, in provider "dbt":
│   10: provider "dbt" {
│ 
│ unexpected end of JSON input
╵
2021-10-27T12:49:15.901+0300 [DEBUG] provider.stdio: received EOF, stopping recv loop: err="rpc error: code = Unavailable desc = transport is closing"
2021-10-27T12:49:15.902+0300 [DEBUG] provider: plugin process exited: path=.terraform/providers/registry.terraform.io/gthesheep/dbt-cloud/0.0.67/linux_amd64/terraform-provider-dbt-cloud_v0.0.67 pid=22052
2021-10-27T12:49:15.902+0300 [DEBUG] provider: plugin exited

[GtheSheep]

Hey, thanks for spotting this!

I've been trying to replicate but haven't quite pinned it yet, but a few things that seemed to set this off for me (bad error handling by me somewhere 😬 )

Just to check, is it the same behaviour if you pass the project_id directly rather than through a var? If you curl the v3 api and pass the project ID at the end of the URL path is the response successful?

I'm not sure if it makes a difference being a personal API token vs a service token from DBT Cloud's auth side of things, but I'll try both later today

Thanks again for spotting!

[stumelius]

@GtheSheep Thanks for the quick reply :)

• Just to check, is it the same behaviour if you pass the project_id directly rather than through a var?: Yes, the same behavior regardless of how I pass the variable

• If you curl the v3 api and pass the project ID at the end of the URL path is the response successful?: Yes, the response is successful with both v2 and v3

curl --request GET \
  --url https://cloud.getdbt.com/api/v3/accounts/$DBT_CLOUD_ACCOUNT_ID/projects/$DBT_PROJECT_ID/ \
  --header "Content-Type: application/json" \
  --header "Authorization: Token $DBT_CLOUD_TOKEN" \
  | jq .

• I'm not sure if it makes a difference being a personal API token vs a service token from DBT Cloud's auth side of things, but I'll try both later today: I tried with both a personal API token and a service token and neither worked :(

[GtheSheep]

Ahh awesome thanks for verifying!

I'm still struggling to reproduce the error tbh, I've cleared all TF artifacts locally, terraform init -upgrade to ensure the pkg is the right version, etc.

I see from your log in the first post that plan seems to complete? So does running plan produce the correct result?

Also, could you let me know what version of Terraform you're using?

[stumelius]

I'm using Terraform v1.0.1. I get the error whenever I try planning or applying - so if I understood your question correctly, the plan doesn't complete but throws an error before it does.

[GtheSheep]

Ahh ok, got it, will test with that version, thanks!

[GtheSheep]

Can't seem to reproduce this on Mac, so have chucked up this PR to test on Linux and still seems to be fine, not sure if cleaning your workspace/ environment may help here? I figured this branch would ensure it's nothing to do with my local setup. https://github.com/GtheSheep/terraform-provider-dbt-cloud/pull/42 I note that this is using env vars over configuring the provider explicitly.

Sorry about this, frustratingly hard to reproduce! 😬 The only time I managed to get this to happen the other day was with some terraform files in the dir (.terraform/ and so on), but can't understand why this would cause issue with the provider itself ☹️

[github-actions[bot]]

This issue is stale because it has been open for 30 days with no activity.

[github-actions[bot]]

This issue was closed because it has been inactive for 7 days since being marked as stale.

[stumelius]

@GtheSheep Getting back to this old issue after a while. Is there any way to get a log trace of the dbt Cloud HTTP requests?

[GtheSheep]

Hey, I've not implemented much logging yet, I guess try setting TF_LOG_LEVEL=TRACE and see what comes out?

[stumelius]

There's nothing related to dbt Cloud HTTP requests in the Terraform trace :(

[ywan160]

I also got the same error when running it on Terraform Cloud.

My TF file as follows:

terraform {
  required_providers {
    dbt = {
      source = "GtheSheep/dbt-cloud"
      version = "0.0.79"
    }
  }
}

provider "dbt" {
  account_id = 12314
  token = "xxxxxxxxxx"
}

resource "dbt_cloud_project" "test_project" {
  name = "test"
}

[GtheSheep]

Hi, @ywan160 - thanks for this, could just check you're definitely using a dbt Cloud service token with (in this case) Account Admin permission set added? As this exact code works for me (running via the cli on Mac OS tbf)

@stumelius - Is the code you're currently trying out the same as initially submitted or the newest provider version?

[ywan160]

Hi, @GtheSheep - Yes, I am using a service token. I had a look at this func - NewClient which is used to initialize the client and ran some tests:

This is the URL for authentication in the code:

url := fmt.Sprintf("%s/v2/accounts/%s", *host_url, strconv.Itoa(*account_id))

If I run curl on this URL format, the body of the web response is empty: Maybe that's what causes the unexpected end of JSON input error.

However, if I add a '/' at the end of the url, it now returns a Json message:

[GtheSheep]

Hey @ywan160 thanks for this, I actually had this issue on my account for some of the other endpoints, let me add this and run tests, if it works I'll release a new provider version to try 🤞

[ywan160]

@GtheSheep Great, thanks. Give me a yell if you need me to run more tests.

[GtheSheep]

Just released v0.0.80 if you could try it plz?

[ywan160]

Unfortunately, still getting the same error. :(

[GtheSheep]

You're doing this in TF Cloud, right? Does it provide any more detailed logging?

[ywan160]

I just used CLI this time. No, it didn't provide more detailed logging:

On Thu, 7 Jul 2022 at 22:18, Gary James @.***> wrote:

You're doing this in TF Cloud, right? Does it provide any more detailed logging?

— Reply to this email directly, view it on GitHub https://github.com/GtheSheep/terraform-provider-dbt-cloud/issues/39#issuecomment-1177361552, or unsubscribe https://github.com/notifications/unsubscribe-auth/AQC6UIZRXMM3XDIIIZXNOALVS2VHJANCNFSM5GZ44DKQ . You are receiving this because you were mentioned.Message ID: @.***>

[GtheSheep]

Are you getting "unexpected end of JSON input" or "Unable to login to DBT Cloud"? Anything you could provide from TF_LOG=TRACE would be super helpful as I can't seem to replicate this 😬

[ywan160]

Hi @GtheSheep, I ran with trace logging but couldn't find anything useful. Unfortunately, the DBT Cloud account that I use is a single-tenant deployment and the log contains some sensitive information so I can't really share it here. :( Did you try it on windows or linux cli instead of Mac to see if you can replicate this?

[GtheSheep]

Ahh that's a pain, how are you setting the account_url in the provider for the single-tenant? Are they of the format https://<name>.getdbt.com/api? I don't know much about how the URLs for those look tbh Thanks so much for all your help on this btw, really appreciate it!

[ywan160]

No problem at all. Yes, it works the same as the shared tenant ones except that you can have your own sub-domain on the URL.

[GtheSheep]

Cool, are you configuring it in the provider definition? Can't see it in the example you sent over is all