API Gateway config should support referencing yaml configs stored in GCP buckets natively

[periwinkleFTW]

Community Note

• Please vote on this issue by adding a 👍 reaction to the original issue to help the community and maintainers prioritize this request
• Please do not leave "+1" or "me too" comments, they generate extra noise for issue followers and do not help prioritize the request
• If you are interested in working on this issue or have submitted a pull request, please leave a comment. If the issue is assigned to the "modular-magician" user, it is either in the process of being autogenerated, or is planned to be autogenerated soon. If the issue is assigned to a user, that user is claiming responsibility for the issue. If the issue is assigned to "hashibot", a community member has claimed the issue already.

Description

I would like to able to deploy API gateway using dynamically generated yaml file that is stored in a GCP bucket. The reason my api config is generated dynamically is because I populate it with links to CloudRun services and Cloud Functions. As it stands right now, deploying such config is complicated. At the plan stage terraform/google_provider evaluates openapi_documents field. This means that I have a dummy api spec in my configuration which is a sensible workaround, but it could be better.

Changing api spec between plan and apply will throw an error about changes to the spec Provider produced an inconsistent result (stuff about config being changed) │ This is a bug in the provider, which should be reported in the provider's │ own issue tracker.. Running apply second time results in a successful deploy because api spec does not change this time around.

What I would really like is to be able to point google_api_gateway_api_config directly to the bucket and the spec file in the bucket. That would make using this resource so much easier.

New or Affected Resource(s)

google_api_gateway_api_config

Potential Terraform Configuration

# Propose what you think the configuration to take advantage of this feature should look like.
# We may not use it verbatim, but it's helpful in understanding your intent.
resource "google_api_gateway_api_config" "api_gw_config" {
  provider      = google-beta
  api           = google_api_gateway_api.api.api_id
  api_config_id = "api-cfg-${var.random_id}"
  project       = var.project
  openapi_documents {
    gcs_storage_document {
      bucket     = var.config_bucket
      name = "api_config/my_config.yaml"
    }
  }
}

References

• 0000

[melinath]

I'm not sure I entirely understand the API gateway side of things but from the TF side, valid configurations should not have a diff after apply. b/284509513

[melinath]

Changing to a bug since it sounds like this represents a "diff after apply" from the user perspective.

[periwinkleFTW]

@melinath and It is kind of a problem for my use case. My API gateway relies on CloudRun urls to properly function. Having to deploy API gateway manually, without it being a part of my terraform configuration is a hassle.

Another issue, at least specifically with GCP is that having deployed API gateway manually in a GCP project managed by terraform will prevent terraform from destroying the project and resources. Terraform throws an error in the middle of destroy operation that it cannot delete child resources. This results in nasty errors that completely breaks the project to the point where it is not possible to delete the project even through console. Now i have zombie projects in my org.

[melinath]

@periwinkleFTW thanks for the additional information! The deletion issue that leaves zombie projects behind sounds pretty bad - but possibly a separate issue from this one? If so it would be great if you could file it separately!

[periwinkleFTW]

@melinath project issue is related to API gateway in my case. This issue pops up only when I deploy API gateway config+gateway manually in my project and then run terraform destroy which tries to destroy all resources and the project. I did not know how to properly submit zombie project issue because I do not want to breed more zombie projects until i find a reliable way to reproduce this error for devs to start working on it.

I have submitted feedback using the pop up when deletion of the project fails. Should I make an issue on github or google issuetracker?

[melinath]

@periwinkleFTW if you could file a new bug issue on the github tracker that lays out the steps you took when you encountered the issue that would be great. It's okay if you aren't sure it will work reliably; I can understand not wanting to reproduce it yourself.

[periwinkleFTW]

I hope that google provider would allow api configs to be changed between plan and apply in case I need to include deployed resources' URLs like CloudFunction and CloudRun in the API Gateway.

As it is designed right now, there is no way to manage compute resources and API gateway in a single terraform configuration which is a shame. CloudRun urls depend on the project they belong to and if I want terraform to deploy whole projects with cloudrun and other compute there is only manual deployment of API config that works.

[periwinkleFTW]

A use-case where a user would want to use GCP resources that provide API endpoints in an API Gateway should me one of the primary use-cases of this resource. As it stands right now api_gateway resource is not very useful in terraform workflows. Current implementation of api_gateway_config and api_gateway_api_gateway require users to deploy vms/cloudrun/cloudfuncs first, get urls and endpoints, make api spec, and then deploy api gateway manually. There is no way to integrate API gateway in a terraform configuration that deploys projects

[KyleWiese]

@periwinkleFTW re the plan -> change -> apply error, do you have a sample flow that is able to repro this?

I've tried from my end by modifying the openapi_documents.document.contents between plan and apply phases but haven't been able to replicate the inconsistent result error you're seeing:

Provider produced an inconsistent result (stuff about config being changed) │ This is a bug in the provider, which should be reported in the provider's │ own issue tracker

[eburgers81]

@KyleWiese I run into this problem at the moment.

I first want to build my openapispec (called spec.yaml) and fill certain values with variable values

resource "local_file" "spec_template" {
  content = templatefile("${path.module}/spec_template.yaml", {
    pre_process_app_url = "${var.pre_process_app_url}"
    }
  )
  filename = "${path.module}/spec.yaml"
}

After this step I deploy my api config based on the just created spec.yaml file.

The terraform plan is working fine, but the terraform apply fails with:

Error: Provider produced inconsistent final plan

When expanding the plan for
module.api-gateway.google_api_gateway_api_config.api-cfg to include new
values learned so far during apply, provider
"registry.terraform.io/hashicorp/google-beta" produced an invalid new value
for .openapi_documents[0].document[0].contents: was

[meer-online]

@eburgers81 Were you able to figure out the solution for this?

[eburgers81]

@eburgers81 Were you able to figure out the solution for this?

Yes I did.

resource "google_api_gateway_api_config" "api_cfg" {
  depends_on              = [google_api_gateway_api.api-gw]
  provider                    = google-beta
  api                             = google_api_gateway_api.api-gw.api_id
  api_config_id_prefix  = var.api_config
  display_name            = var.api_cfg_display_name

  openapi_documents {
    document {
      path = "${var.api_spec_template}_${var.environment}.yaml"
      contents = base64encode(templatefile("${path.module}/${var.api_spec_template}.yaml", {
        pre_process_app_url = var.pre_process_app_url
      }))
    }
  }
  lifecycle {
    create_before_destroy = true
  }

  gateway_config {
    backend_config {
      google_service_account = var.service_account_email
    }
  }
}