google_bigquery_dataset import failure

[antoinedeschenes]

Community Note

• Please vote on this issue by adding a 👍 reaction to the original issue to help the community and maintainers prioritize this request.
• Please do not leave +1 or me too comments, they generate extra noise for issue followers and do not help prioritize the request.
• If you are interested in working on this issue or have submitted a pull request, please leave a comment.
• If an issue is assigned to the modular-magician user, it is either in the process of being autogenerated, or is planned to be autogenerated soon. If an issue is assigned to a user, that user is claiming responsibility for the issue. If an issue is assigned to hashibot, a community member has claimed the issue already.

Terraform Version

Terraform v1.6.6
on darwin_amd64
+ provider registry.terraform.io/hashicorp/google v5.11.0

Affected Resource(s)

• google_bigquery_dataset

Terraform Configuration Files

# imports.tf
import {
  id = "projects/myproject/datasets/mydataset"
  to = google_bigquery_dataset.main
}

# generated.tf
# __generated__ by Terraform
resource "google_bigquery_dataset" "main" {
  dataset_id                      = "DATASET"
  default_collation               = null
  default_partition_expiration_ms = null
  default_table_expiration_ms     = null
  delete_contents_on_destroy      = false
  description                     = null
  friendly_name                   = null
  is_case_insensitive             = false
  labels                          = {}
  location                        = "US"
  max_time_travel_hours           = null
  project                         = "PROJECT"
  storage_billing_model           = null
  access {
    domain         = null
    group_by_email = null
    iam_member     = null
    role           = "OWNER"
    special_group  = null
    user_by_email  = "ACCOUNT@PROJECT.iam.gserviceaccount.com"
  }
  access {
    domain         = null
    group_by_email = null
    iam_member     = null
    role           = "WRITER"
    special_group  = null
    user_by_email  = "ACCOUNT@PROJECT.iam.gserviceaccount.com"
  }
  timeouts {
    create = null
    delete = null
    update = null
  }
}

Debug Output

https://gist.github.com/antoinedeschenes/7b380ed9c0a38b86b8ebd369bc8edcb5

Panic Output

Expected Behavior

Plan command should exit successfully

╷
│ Warning: Config generation is experimental
│ 
│ Generating configuration during import is currently experimental, and the generated configuration format may change in future versions.
╵

Actual Behavior

Plan crashes with

Planning failed. Terraform encountered an error while generating this plan.

╷
│ Warning: Config generation is experimental
│ 
│ Generating configuration during import is currently experimental, and the generated configuration format may change in future versions.
╵
╷
│ Error: "default_table_expiration_ms" cannot be shorter than 3600000 milliseconds (one hour)
│ 
│   with google_bigquery_dataset.main,
│   on generated.tf line 4:
│   (source code not available)
│ 
╵

Steps to Reproduce

1. create above imports.tf terraform file only.
2. terraform init
3. terraform plan -generate-config-out=generated.tf

Important Factoids

• Using an import block to generate config
• The imported values are optional in the resource documentation. Planning works properly if these values are manually set to null. The validateDefaultTableExpirationMs should probably ignore a 0 value as it's a valid value in the BigQuery API

  Optional. The default lifetime of all tables in the dataset, in milliseconds. The minimum lifetime value is 3600000 milliseconds (one hour). To clear an existing default expiration with a PATCH request, set to 0.

References

• https://registry.terraform.io/providers/hashicorp/google/latest/docs/resources/bigtable_table_iam#import
• https://github.com/hashicorp/terraform-provider-google/blob/release-5.11.0/google/services/bigquery/resource_bigquery_dataset.go#L53C6-L53C38
• https://cloud.google.com/bigquery/docs/reference/rest/v2/datasets#Dataset:~:text=of%20the%20dataset.-,defaultTableExpirationMs,-string%20(Int64Value)

b/319825217

[edwardmedia]

@antoinedeschenes it works for me. Can you try similar?

resource "google_bigquery_dataset" "main" {
    dataset_id = 1234
}

import {
  id = "projects/myprojects/datasets/imydataset"
  to = google_bigquery_dataset.main
}

resource "google_bigquery_dataset" "main" {
    dataset_id = 1234
}

import {
  id = "myprojects/mydataset"
  to = google_bigquery_dataset.main
}

[antoinedeschenes]

@edwardmedia don't define a resource in the terraform file, terraform is supposed to generate it with the following command: terraform plan -generate-config-out=generated.tf

---

1. Create blank folder
2. add only the following code:

   import {
    id = "myprojects/mydataset"
    to = google_bigquery_dataset.main
   }

3. terraform init && terraform plan -generate-config-out=generated.tf
4. generated.tf is created successfully, but doesn't pass validation if the dataset's default_table_expiration_ms is set to 0/disabled

[edwardmedia]

@antoinedeschenes can you share the debug log for your import?

[antoinedeschenes]

@edwardmedia there's the log: https://gist.github.com/antoinedeschenes/7b380ed9c0a38b86b8ebd369bc8edcb5

My issue is that the generated terraform contains default_table_expiration_ms = 0 in the resource attributes. It's a valid value for the API, but not the validation code. The input validation code expects null and translates it to a 0 in the resource state

[edwardmedia]

Yes, I can repro it and see the error.

Questionable function is here

[antoinedeschenes]

Looking here https://cloud.google.com/bigquery/docs/reference/rest/v2/datasets#Dataset it looks like 0 is a valid value