BigQuery Salesforce Transfer | Requested entity was not found

RomeoAva commented 3 months ago

Community Note

Please vote on this issue by adding a 👍 reaction to the original issue to help the community and maintainers prioritize this request.
Please do not leave +1 or me too comments, they generate extra noise for issue followers and do not help prioritize the request.
If you are interested in working on this issue or have submitted a pull request, please leave a comment.
If an issue is assigned to a user, that user is claiming responsibility for the issue.
Customers working with a Google Technical Account Manager or Customer Engineer can ask them to reach out internally to expedite investigation and resolution of this issue.

Terraform Version & Provider Version(s)

terraform version Terraform v1.5.7 on darwin_arm64

provider registry.terraform.io/hashicorp/google v5.40.0

Affected Resource(s)

After this fix has been pushed: https://github.com/hashicorp/terraform-provider-google/issues/18506

I went ahead and upgrade my Terraform to use v5.40.0 of Google provider.

Terraform Configuration

resource "google_bigquery_data_transfer_config" "salesforce_account_import" {
  display_name           = "Salesforce Account Import"
  location               = "US"
  data_source_id         = "accounts"
  destination_dataset_id = "analysis"
  project                = var.projects[terraform.workspace]
  params = {
    "connector.authentication.oauth.clientId"     = data.google_secret_manager_secret_version.salesforce_consumer_key_version.secret_data
    "connector.authentication.oauth.clientSecret" = data.google_secret_manager_secret_version.salesforce_consumer_secret_version.secret_data
    "connector.authentication.oauth.myDomain"     = var.salesforce_url[terraform.workspace]
    "assets"                                      = "[Account]"
  }
}

Debug Output

google_bigquery_data_transfer_config.salesforce_account_import: Creating...
╷
│ Error: Error creating Config: googleapi: Error 404: Requested entity was not found.
│ 
│   with google_bigquery_data_transfer_config.salesforce_account_import,
│   on bigquery_transfers.tf line 13, in resource "google_bigquery_data_transfer_config" "salesforce_account_import":
│   13: resource "google_bigquery_data_transfer_config" "salesforce_account_import" {

Expected Behavior

To create the BQ Transfer

Actual Behavior

Errors out. I think a point to look at is that between the first issue I posted and this one Google's documenation on how to create a BQ Salesforce Transfer has changed: https://cloud.google.com/bigquery/docs/salesforce-transfer

You no longer Salesforce Username/Password and Security Token

Maybe they also changed their backend URL to create a new transfer?

Steps to reproduce

No response

Important Factoids

No response

References

No response

ggtisc commented 2 months ago

Hi @RomeoAva!

You are getting this error because you are trying to use an invalid value for data_source_id.

According to the documentation there isn't any data source called accounts for google_bigquery_data_transfer_config.

it's not possible to use a custom data source directly with the google_bigquery_data_transfer_config resource in Terraform.

BigQuery Data Transfer Service provides a predefined set of connectors for integrating with various data sources. These connectors are built and maintained by Google to ensure compatibility and reliability.

Here's the official Google Cloud documentation link where you can find a comprehensive list of supported data sources for the BigQuery Data Transfer Service

Finally I suggest you read the terraform registry and API documentation before you start using this resource because there are a set of rules you may take in consideration.

RomeoAva commented 2 months ago

Hi @RomeoAva!

You are getting this error because you are trying to use an invalid value for data_source_id.

According to the documentation there isn't any data source called accounts for google_bigquery_data_transfer_config.

it's not possible to use a custom data source directly with the google_bigquery_data_transfer_config resource in Terraform.

BigQuery Data Transfer Service provides a predefined set of connectors for integrating with various data sources. These connectors are built and maintained by Google to ensure compatibility and reliability.

Here's the official Google Cloud documentation link where you can find a comprehensive list of supported data sources for the BigQuery Data Transfer Service

Finally I suggest you read the terraform registry and API documentation before you start using this resource because there are a set of rules you may take in consideration.

Aaaaahh silly me. I misunderstood the purpose of data_source_id. Thank you for the clarifications again!

github-actions[bot] commented 1 month ago

I'm going to lock this issue because it has been closed for 30 days ⏳. This helps our maintainers find and focus on the active issues. If you have found a problem that seems similar to this, please open a new issue and complete the issue template so we can capture all the details necessary to investigate further.

hashicorp / terraform-provider-google