Add resources to validate a domain

toadjaune commented 4 years ago

Community Note

Please vote on this issue by adding a 👍 reaction to the original issue to help the community and maintainers prioritize this request
Please do not leave "+1" or "me too" comments, they generate extra noise for issue followers and do not help prioritize the request
If you are interested in working on this issue or have submitted a pull request, please leave a comment. If the issue is assigned to the "modular-magician" user, it is either in the process of being autogenerated, or is planned to be autogenerated soon. If the issue is assigned to a user, that user is claiming responsibility for the issue. If the issue is assigned to "hashibot", a community member has claimed the issue already.

Description

Several operations on Google Cloud require you to validate domain ownership. For example, creating a GCS bucket for static website hosting. This is the usecase I met and the only one I know of, I suppose however there should be other affected resources.

There are currently no terraform resources for automating the validation process, nor to delegate domain ownership to a service account, which has several implications :

You need to manually validate the domain before being able to automatically create some resources. This can be an important source of friction for the operator, especially if the validation relies on resources created by terraform (say, registering a domain …). You would then either need to first run your resource creation, have it fail in the middle, manually validate, then restart your resource creation. Or structuring it differently. Either way, there's a manual step and extra complexity.
As far as I can tell, there is no way with standard tooling to validate domain ownership directly with a service account, meaning that if you want to run terraform from a service account you need to manually validate from an actual user account and then delegate ownership to this service account (using the old webmaster interface, the new one does not offer this option). And if your personal user account is ever disabled, this delegation apparently goes away too.

TL;DR : a lot of friction and manual work no matter what you do.

There is an API for automating site verification. I don't know if it can be used when authenticated as a service account, but even if not, being able to automate this would be a great step forward.

New or Affected Resource(s)

AWS have several resources requiring similar validation processes, my API proposal is based on the one used by aws_ses_domain_identity_verification

google_domain_owner_challenge
google_domain_owner_validation
google_domain_owner

Potential Terraform Configuration

# Request domain ownership.
# This resource returns a TXT record that you will need to publish in order for google
# to recognize you as an owner
resource "google_domain_owner_challenge" "example" {
  domain = "example.com"
}

# Publish the DNS record for the challenge
# This does not need to be google, any managed DNS would do the job
resource "google_dns_record_set" "challenge" {
  name = google_domain_owner_challenge.example.challenge.name
  type = "TXT"
  ttl  = 300

  # Of course, you would need to define this zone
  managed_zone = google_dns_managed_zone.example.name

  rrdatas = [google_domain_owner_challenge.example.challenge.value]
}

# This resource does not actually create anything on the google side.
# It's there to wait for the domain validation to be accepted by google
# (DNS propagation, etc…) before doing anything actually requiring
# the corresponding permissions
resource "google_domain_owner_validation" "example" {
  name = "example.com"
  depends_on = [google_dns_record_set.challenge]
}

# Create a bucket with the previously registered domain
# Please note the implicit dependency on the validation object
resource "google_storage_bucket" "static_front_test" {
  name = google_domain_owner_validation.example.name
  website {
    main_page_suffix = "index.html"
    not_found_page   = "index.html"
  }
}

# Delegate ownership to another user
# That could be a human administrator that needs to manage SEO in the search console…
# Please note the implicit dependency on the validation object
resource "google_domain_owner" "example_webmaster" {
  domain = google_domain_owner_validation.example.name
  user = "webmaster@example.com"
}

References

Relevant documentation was referenced above

hectorj commented 4 years ago

Similar issue was closed before: https://github.com/terraform-providers/terraform-provider-google/issues/1724

But I think it would be a good addition to this provider!

In the meantime, you can use this: https://github.com/hectorj/terraform-provider-googlesiteverification

Edit: I'm interested in trying to contribute it to this repo. I'll take a shot at it soon-ish. Edit2: I do not have much free time for this at the moment. If someone else feels like taking a shot at it, do not hesitate.

mvanholsteijn commented 4 years ago

@danawillow @hectorj and I are perfectly willing to assist in providing the required resources. If you feel it can be auto-generated, then that would be awesome. I do think it needs some work as the API is not orthogonal. ie. the create and update have a different signature.

danawillow commented 4 years ago

If you already have handwritten code for the resource, then it's not mandatory to do it generated, especially if it's unusual. The one thing I would be very careful about is authentication- our guideline right now is that you should be able to use the same authentication method for all your resources, and based on https://github.com/hashicorp/terraform-provider-google/issues/1724#issuecomment-403012664 I'm not 100% sure you can.

mvanholsteijn commented 4 years ago

@hectorj actually programmed it to work properly; if you look at the code at https://github.com/hectorj/terraform-provider-googlesiteverification, you can specify JSON service account key to authenticate with and that works. the site verification API did not work with the default Oauth 2.0 credentials, I actually created a separate service account to verify the domain with. See:

So this issue can be documented or even resolved in the resource definition.

voycey commented 1 year ago

This is absolutely required from an automation stand point and I am actually incredulous that this isn't standard in the provider or that the APIs aren't set up in such a way at GCP.

The above module doesn't work (at least not with latest versions of Terraform) and there is no documentation - please get this supported in the provider asap

TrieBr commented 1 year ago

At 2 years old and 44 upvotes, is there any progress on this or plans to add it? This totally breaks automation when creating GCS buckets. I managed to create a workaround which manually invokes the site verification APIs through http data requests. The I use the following module to verify a domain. The downside is that the access token appears in the logs, but using a short lived one (2s) should help.

variable "project_id" { type = string }
variable "dns_name" { type = string }
variable "managed_dns_zone" { type = string }
variable "service_account_email" { type = string }
// Create an access token.
data "google_service_account_access_token" "verification" {
  provider               = google
  target_service_account = "${var.service_account_email}"
  scopes                 = ["https://www.googleapis.com/auth/siteverification", "https://www.googleapis.com/auth/siteverification.verify_only"]
  lifetime               = "2s"
}

// Invokes the Site Verification API to get a token.
// https://developers.google.com/site-verification/v1/invoking#verify
data "http" "api_verification" {
  url = "https://www.googleapis.com/siteVerification/v1/token?access_token=${data.google_service_account_access_token.verification.access_token}"
  method = "POST"
  request_body = <<EOT
                {
                "verificationMethod": "DNS_TXT",
                "site": {
                  "identifier": "${var.dns_name}",
                  "type": "INET_DOMAIN"
                }
              }
              EOT

  # Optional request headers
  request_headers = {
    Content-Type = "application/json"
  }
  lifecycle {
    postcondition {
      condition     = contains([201, 204, 200], self.status_code)
      error_message = self.body
    }
  }
  depends_on = [
    data.google_service_account_access_token.verification
  ]
}

// Domain verification for static storage url.
resource "google_dns_record_set" "storage_static_verification" {
  project     = var.project_id
  name = var.dns_name
  type = "TXT"
  ttl  = 300
  managed_zone = var.managed_dns_zone
  rrdatas = [jsondecode(data.http.api_verification.response_body)["token"]]
  depends_on = [data.http.api_verification]
}

data "google_service_account_access_token" "verification_confirm" {
  provider               = google
  target_service_account = "${var.service_account_email}"
  scopes                 = ["https://www.googleapis.com/auth/siteverification", "https://www.googleapis.com/auth/siteverification.verify_only"]
  lifetime               = "2s"
}
// From https://developers.google.com/site-verification/v1/invoking#exampleInsert
data "http" "api_verification_insert" {
  url = "https://www.googleapis.com/siteVerification/v1/webResource?verificationMethod=DNS_TXT&access_token=${data.google_service_account_access_token.verification_confirm.access_token}"
  method = "POST"
  request_body = <<EOT
                {
                "site": {
                  "identifier": "${var.dns_name}",
                  "type": "INET_DOMAIN"
                }
              }
              EOT

  request_headers = {
    Content-Type = "application/json"
  }
  lifecycle {
    postcondition {
      condition     = contains([201, 204, 200], self.status_code)
      error_message = self.body
    }
  }
  depends_on = [
    data.google_service_account_access_token.verification_confirm
  ]
}

sreid99 commented 1 year ago

Any chance this could be added ? Ideally with OpenID Connect as the authentication method.

melinath commented 6 months ago

I was able to confirm that this API can use the same OAuth 2.0 credentials that the Terraform provider operates on, so this seems like it would make sense in the google and google-beta providers.

SirGitsalot commented 5 months ago

👋

andromaqui commented 5 months ago

We are also facing the same issue currently - a fix would be much apppreciated

hashicorp / terraform-provider-google