edwardmedia
commented
4 years ago @kikuingithub can you post the debug log so I can take a close look?
Closed kikuingithub closed 4 years ago
edwardmedia
commented
4 years ago @kikuingithub can you post the debug log so I can take a close look?
kikuingithub
commented
4 years ago Hi Edward,
Thanks for the quick turnaround ! Here are the debug logs.
2020/06/12 20:36:52 [TRACE] Executing graph transform
terraform.OrphanResourceCountTransformer
2020/06/12 20:36:52 [TRACE] Completed graph transform
terraform.OrphanResourceCountTransformer (no changes)
2020/06/12 20:36:52 [TRACE] Executing graph transform
terraform.AttachStateTransformer
2020/06/12 20:36:52 [DEBUG] Resource instance state not found for node
"google_composer_environment.dwh-composer-7te458r", instance
google_composer_environment.dwh-composer-7te458r
2020/06/12 20:36:52 [TRACE] Completed graph transform
terraform.AttachStateTransformer (no changes)
2020/06/12 20:36:52 [TRACE] Executing graph transform
terraform.TargetsTransformer
2020/06/12 20:36:52 [TRACE] Completed graph transform
terraform.TargetsTransformer (no changes)
2020/06/12 20:36:52 [TRACE] Executing graph transform
terraform.ReferenceTransformer
2020/06/12 20:36:52 [DEBUG] ReferenceTransformer:
"google_composer_environment.dwh-composer-7te458r" references: []
2020/06/12 20:36:52 [TRACE] Completed graph transform
terraform.ReferenceTransformer (no changes)
2020/06/12 20:36:52 [TRACE] Executing graph transform
terraform.RootTransformer
2020/06/12 20:36:52 [TRACE] Completed graph transform
terraform.RootTransformer (no changes)
2020/06/12 20:36:52 [TRACE] vertex
"google_composer_environment.dwh-composer-7te458r": entering dynamic
subgraph
2020/06/12 20:36:52 [TRACE] dag/walk: updating graph
2020/06/12 20:36:52 [TRACE] dag/walk: added new vertex:
"google_composer_environment.dwh-composer-7te458r"
2020/06/12 20:36:52 [TRACE] dag/walk: visiting
"google_composer_environment.dwh-composer-7te458r"
2020/06/12 20:36:52 [TRACE] vertex
"google_composer_environment.dwh-composer-7te458r": starting visit
(terraform.NodePlannableResourceInstance)
2020/06/12 20:36:52 [TRACE] vertex
"google_composer_environment.dwh-composer-7te458r": evaluating
2020/06/12 20:36:52 [TRACE] [walkPlan] Entering eval tree:
google_composer_environment.dwh-composer-7te458r
2020/06/12 20:36:52 [TRACE]
"config.0.node_config.0.ip_allocation_policy.0.services_secondary_range_name": conflicts with config.0.node_config.0.ip_allocation_policy.0.services_ipv4_cidr_block
"config.0.node_config.0.ip_allocation_policy.0.cluster_ipv4_cidr_block": conflicts with config.0.node_config.0.ip_allocation_policy.0.cluster_secondary_range_name
"config.0.node_config.0.ip_allocation_policy.0.services_ipv4_cidr_block":
conflicts with
config.0.node_config.0.ip_allocation_policy.0.services_secondary_range_name
2020/06/12 20:36:52 [ERROR]
"config.0.node_config.0.ip_allocation_policy.0.cluster_secondary_range_name": conflicts with config.0.node_config.0.ip_allocation_policy.0.cluster_ipv4_cidr_block
"config.0.node_config.0.ip_allocation_policy.0.services_secondary_range_name": conflicts with config.0.node_config.0.ip_allocation_policy.0.services_ipv4_cidr_block
"config.0.node_config.0.ip_allocation_policy.0.cluster_ipv4_cidr_block": conflicts with config.0.node_config.0.ip_allocation_policy.0.cluster_secondary_range_name
"config.0.node_config.0.ip_allocation_policy.0.services_ipv4_cidr_block": conflicts with config.0.node_config.0.ip_allocation_policy.0.services_secondary_range_name 2020/06/12 20:36:52 [TRACE] [walkPlan] Exiting eval tree: google_composer_environment.dwh-composer-7te458r 2020/06/12 20:36:52 [TRACE] vertex "google_composer_environment.dwh-composer-7te458r": visit complete 2020/06/12 20:36:52 [TRACE] vertex "google_composer_environment.dwh-composer-7te458r": dynamic subgraph encountered errors 2020/06/12 20:36:52 [TRACE] vertex "google_composer_environment.dwh-composer-7te458r": visit complete 2020/06/12 20:36:52 [TRACE] dag/walk: upstream of "output.airflow_uri" errored, so skipping 2020/06/12 20:36:52 [TRACE] dag/walk: upstream of "provider.google (close)" errored, so skipping 2020/06/12 20:36:52 [TRACE] dag/walk: upstream of "meta.count-boundary (EachMode fixup)" errored, so skipping 2020/06/12 20:36:52 [TRACE] dag/walk: upstream of "root" errored, so skipping 2020/06/12 20:36:52 [INFO] backend/local: plan operation completed 2020/06/12 20:36:52 [TRACE] statemgr.Filesystem: removing lock metadata file .terraform.tfstate.lock.info 2020/06/12 20:36:52 [TRACE] statemgr.Filesystem: unlocked by closing terraform.tfstate
Let me know if more info is needed on this.
On Fri, Jun 12, 2020 at 8:36 PM Edward Sun notifications@github.com wrote:
@kikuingithub https://github.com/kikuingithub can you post the debug log so I can take a close look?
— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/terraform-providers/terraform-provider-google/issues/6579#issuecomment-643323076, or unsubscribe https://github.com/notifications/unsubscribe-auth/AELFFSVKTGD63TQRQIKRAITRWI777ANCNFSM4N4HFKUA .
edwardmedia
commented
4 years ago @kikuingithub can you add steps to repro the issue?
kikuingithub
commented
4 years ago Nothing significant..!
terraform init .. plan.. apply. The resource declaration for google_composer_envoronment is included in the github issue details. I would also like to add that the terraform doc for the composer says that any two of the four attributes in the ip_allocation_policy block are required. But the actually, all four attribites are mandatory.
Thanks,
On Fri, Jun 12, 2020, 9:12 PM Edward Sun notifications@github.com wrote:
@kikuingithub https://github.com/kikuingithub can you add steps to repro the issue?
— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/terraform-providers/terraform-provider-google/issues/6579#issuecomment-643342168, or unsubscribe https://github.com/notifications/unsubscribe-auth/AELFFSRDWK5KMV2GJZYOPWTRWJEHNANCNFSM4N4HFKUA .
edwardmedia
commented
4 years ago @kikuingithub can you share the plan?
edwardmedia
commented
4 years ago The doc is correct, according to https://cloud.google.com/composer/docs/reference/rest/v1/projects.locations.environments#ipallocationpolicy. We should only accept two
kikuingithub
commented
4 years ago This will never take me to plan. The terraform execution stops with the ip_allocation conflict message which i have already posted.
On Fri, Jun 12, 2020 at 9:58 PM Edward Sun notifications@github.com wrote:
@kikuingithub https://github.com/kikuingithub can you share the plan?
— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/terraform-providers/terraform-provider-google/issues/6579#issuecomment-643370210, or unsubscribe https://github.com/notifications/unsubscribe-auth/AELFFSQYOUDEP6AIHZKLU3LRWJJSPANCNFSM4N4HFKUA .
kikuingithub
commented
4 years ago Hi Edward, Hope you are doing well !!
Do you need any more information from me? I am happy to provide.
Found that shared VPC is NOT supported by the V1 version of API,
but it is implemented in v1beta1. Is this beta API version used in the
latest terraform modules ?
Thanks !!
On Fri, Jun 12, 2020 at 10:25 PM Edward Sun notifications@github.com wrote:
The doc is correct, according to https://cloud.google.com/composer/docs/reference/rest/v1/projects.locations.environments#ipallocationpolicy
— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/terraform-providers/terraform-provider-google/issues/6579#issuecomment-643382343, or unsubscribe https://github.com/notifications/unsubscribe-auth/AELFFSRUGQXX2D5KVHPMLOTRWJMYVANCNFSM4N4HFKUA .
emilymye
commented
4 years ago @kikuingithub May I ask where you see that all ip_allocation_policy attributes are required? From our docs and the GCP docs, you can only specify cluster or service IP addresses by one of _secondary_range_name or _ipv4_cidr_block inside this block. i.e. valid configs would be:
ip_allocation_policy {
use_ip_aliases = true
cluster_secondary_range_name = "..."
service_secondary_range_name = "..."
}or
ip_allocation_policy {
use_ip_aliases = true
cluster_ipv4_cidr_block = "..."
service_ipv4_cidr_block = "..."
}That's what the conflicting messages mean.
kikuingithub
commented
4 years ago Hi Edward,
Replying your previous query, ip_allocation_policy { cluster_ipv4_cidr_block = "" cluster_secondary_range_name = "name of range1" services_ipv4_cidr_block = "" services_secondary_range_name = "name of range 2" use_ip_aliases = true }
This is not liked by Terraform and throws the previous stated error that all fields are mandatory.
Can you please clear me if the current google_composer_environment terraform module supports composer environment with shared VPC (that implements v1beta1 of google rest API). Please provide me the versions of the module that would help me in understanding it better.
Thanks !!
On Mon, Jun 15, 2020 at 11:02 PM emily notifications@github.com wrote:
@kikuingithub https://github.com/kikuingithub May I ask where you see that ip_allocation_policy attributes are required? From our (docs)[ https://www.terraform.io/docs/providers/google/r/composer_environment.html] and the GCP docs https://cloud.google.com/composer/docs/reference/rest/v1/projects.locations.environments#ipallocationpolicy, you can only specify cluster or service IP addresses by one of _secondary_range_name or _ipv4_cidr_block inside this block. i.e. valid configs would be:
ip_allocation_policy { use_ip_aliases = true cluster_secondary_range_name = "..." service_secondary_range_name = "..." }
or
ip_allocation_policy { use_ip_aliases = true cluster_ipv4_cidr_block = "..." service_ipv4_cidr_block = "..." }
— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/terraform-providers/terraform-provider-google/issues/6579#issuecomment-644270034, or unsubscribe https://github.com/notifications/unsubscribe-auth/AELFFSW2TUD2RLUPWMVKWB3RWZLMRANCNFSM4N4HFKUA .
emilymye
commented
4 years ago Hi @kikuingithub - I can't seem to find the error message you mentioned previously about all fields being mandatory. I only see the conflicts_with error message when you supply all four optional nested fields in ip_allocation_policy. Did you post a log with that message and the exact config?
Shared VPC (ip_allocation_policy fields) was added to the GA provider in 2.12.0 (marked as private environment config), so it actually is supported for both providers. The GKE (Container Cluster) v1beta1 API is the beta-only API
As an example of a working shared VPC config, I am able to successfully run this config on both the GA and beta provider. Note that the ip_allocation_policy has ONLY cluster_secondary_range_name and services_secondary_range_name specified.
resource "google_compute_network" "vpc" {
name = "sharedvpc"
auto_create_subnetworks = false
}
resource "google_compute_subnetwork" "vpc-sub" {
name = google_compute_network.vpc.name
network = google_compute_network.vpc.name
ip_cidr_range = "10.0.36.0/24"
region = "us-central1"
private_ip_google_access = true
secondary_ip_range {
range_name = "pod"
ip_cidr_range = "10.0.0.0/19"
}
secondary_ip_range {
range_name = "svc"
ip_cidr_range = "10.0.32.0/22"
}
}
resource "google_composer_environment" "test" {
name = "test-env"
region = "us-central1"
config {
node_config {
network = google_compute_network.vpc.self_link
subnetwork = google_compute_subnetwork.vpc-sub.self_link
zone = "us-central1-a"
ip_allocation_policy {
use_ip_aliases = true
cluster_secondary_range_name = google_compute_subnetwork.test.secondary_ip_range[0].range_name
services_secondary_range_name = google_compute_subnetwork.test.secondary_ip_range[1].range_name
}
}
}
}Based on the example given in the previous reply, you are still including cluster_ipv4_cidr_block and services_ipv4_cidr_block. Either you need to remove these two fields or add them and remove the secondary_range_name fields.
kikuingithub
commented
4 years ago Hi ,
In above example, can you try it with actual shared VPC, that is shared
from a host project..
Yes, the conflict error is what i see when I give all the attribues in the IP_allocation_policy, it worked when i give only range names or address ranges. And now i am getting a new error. But when tried from Terraform getting below error. CREATE operation on this environment failed 51 minutes ago with the following error message: Http error status code: 400 Http error message: BAD REQUEST Additional errors: {"ResourceType":"serviceproject1/us-central1-dev-dwh-dwh-com-05e4dedf-addons-gke-typer:container.projects.zones.clusters.nodePools.setManagement","ResourceErrorCode":"400","ResourceErrorMessage":{"code":400,"message":"Cluster is currently being created, deleted, updated or repaired and cannot be updated.","status":"FAILED_PRECONDITION","details":[],"statusMessage":"Bad Request","requestPath":" https://container.googleapis.com/v1/projects/serviceproject1/zones/us-central1-b/clusters/us-central1-dev-dwh-dwh-com-05e4dedf-gke/nodePools/default-pool/setManagement ","httpMethod":"POST"}}
I am able to use shared VPC and create the composer environment from UI. Are you sure that the current terraform module supports Shared VPC.
Thanks,
On Tue, Jun 16, 2020 at 11:29 PM emily notifications@github.com wrote:
Hi @kikuingithub https://github.com/kikuingithub - I can't seem to find the error message you mentioned previously about all fields being mandatory. I only see the conflicts_with error message when you supply all four optional nested fields in ip_allocation_policy. Did you post a log with that message and the exact config?
Shared VPC (ip_allocation_policy fields) was added to the GA provider in 2.12.0 https://github.com/terraform-providers/terraform-provider-google/blob/master/CHANGELOG.md#2120-august-01-2019 (marked as private environment config), so it actually is supported for both providers. The GKE (Container Cluster) v1beta1 API is the beta-only API
As an example of a working shared VPC config, I am able to successfully run this config on both the GA and beta provider. Note that the ip_allocation_policy has ONLY cluster_secondary_range_name and services_secondary_range_name specified.
resource "google_compute_network" "vpc" { name = "sharedvpc" auto_create_subnetworks = false }
resource "google_compute_subnetwork" "vpc-sub" { name = google_compute_network.vpc.name network = google_compute_network.vpc.name ip_cidr_range = "10.0.36.0/24" region = "us-central1" private_ip_google_access = true
secondary_ip_range { range_name = "pod" ip_cidr_range = "10.0.0.0/19" }
secondary_ip_range { range_name = "svc" ip_cidr_range = "10.0.32.0/22" } }
resource "google_composer_environment" "test" { name = "test-env" region = "us-central1" config { node_config { network = google_compute_network.vpc.self_link subnetwork = google_compute_subnetwork.vpc-sub.self_link zone = "us-central1-a"
ip_allocation_policy { use_ip_aliases = true cluster_secondary_range_name = google_compute_subnetwork.test.secondary_ip_range[0].range_name services_secondary_range_name = google_compute_subnetwork.test.secondary_ip_range[1].range_name } }} }
Based on the example given in the previous reply, you are still including cluster_ipv4_cidr_block and services_ipv4_cidr_block. Either you need to remove these two fields or add them and remove the secondary_range_name fields.
— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/terraform-providers/terraform-provider-google/issues/6579#issuecomment-644919723, or unsubscribe https://github.com/notifications/unsubscribe-auth/AELFFSRT2F5Y77DYAO4AICLRW6XJHANCNFSM4N4HFKUA .
emilymye
commented
4 years ago The error you got on creating the environment after 51 minutes actually looks like the creation API call succeeded but the Composer API returns an operation error because the underlying GKE cluster is being edited.
At this point, this is an upstream API error (i.e. a Composer issue, not a Terraform one) and likely a temporary timing issue. I can file a bug against the upstream API to handle issues like this better (they should wait on cluster updates if something else is happening concurrently to the cluster)
Otherwise, all we can really do is just try the same operation again. I understand it's very frustrating because it takes SO LONG, but this is a pretty complex resource that spins up several different GCP resources in the background and a lot of those background requests can fail and get returned as non-retryable errors by the Composer API. We encounter errors like this fairly regularly during our tests.
As for shared VPC support, support in the Terraform provider really means "can ip_allocation_policy be set". If you look at your debug logs, you should be able to see that the request is sent with the ip_allocation_policy fields, so we support shared VPC in both providers.
I'd also caution against using the word "module" to talk about the provider, since Terraform modules mean a very different thing than the provider.
kikuingithub
commented
4 years ago Hi,
Yes please .. do raise a bug against the upstream API and share the details here so that i can follow up on it.
Once again, Thanks for all your help !!
Thanks,
On Thu, Jun 18, 2020, 12:16 AM emily notifications@github.com wrote:
The error you got on creating the environment after 51 minutes actually looks like the creation API call succeeded but the Composer API returns an operation error because the underlying GKE cluster is being edited.
At this point, this is an upstream API error (i.e. a Composer issue, not a Terraform one) and likely a temporary timing issue. I can file a bug against the upstream API to handle issues like this better (they should wait on cluster updates if something else is happening concurrently to the cluster)
Otherwise, all we can really do is just try the same operation again. I understand it's very frustrating because it takes SO LONG, but this is a pretty complex resource that spins up several different GCP resources in the background and a lot of those background requests can fail and get returned as non-retryable errors by the Composer API. We encounter errors like this fairly regularly during our tests.
As for shared VPC support, support in the Terraform provider really means "can ip_allocation_policy be set". If you look at your debug logs, you should be able to see that the request is sent with the ip_allocation_policy fields, so we support shared VPC in both providers.
I'd also caution against using the word "module" to talk about the provider, since Terraform modules https://www.terraform.io/docs/modules/index.html mean a very different thing than the provider.
— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/terraform-providers/terraform-provider-google/issues/6579#issuecomment-645554432, or unsubscribe https://github.com/notifications/unsubscribe-auth/AELFFSUOAL5W4XD4C34EKPLRXEFQZANCNFSM4N4HFKUA .
ghost
commented
4 years ago I'm going to lock this issue because it has been closed for 30 days ⏳. This helps our maintainers find and focus on the active issues.
If you feel this issue should be reopened, we encourage creating a new issue linking back to this one for added context. If you feel I made an error 🤖 🙉 , please reach out to my human friends 👉 hashibot-feedback@hashicorp.com. Thanks!
Community Note
modular-magicianuser, it is either in the process of being autogenerated, or is planned to be autogenerated soon. If an issue is assigned to a user, that user is claiming responsibility for the issue. If an issue is assigned tohashibot, a community member has claimed the issue already.Terraform Version - 0.12.20
Affected Resource(s)
Terraform Configuration Files
Debug Output
Panic Output
Expected Behavior
cloud composer should accept the ip_allocation_policy block attributes and should create the composer environment.
Actual Behavior
Error: "config.0.node_config.0.ip_allocation_policy.0.cluster_secondary_range_name": conflicts with config.0.node_config.0.ip_allocation_policy.0.cluster_ipv4_cidr_block
on main.tf line 45, in resource "google_composer_environment" "dwh-composer-XXXr": 45: resource "google_composer_environment" "dwh-composer-XXXr" {
Error: "config.0.node_config.0.ip_allocation_policy.0.services_secondary_range_name": conflicts with config.0.node_config.0.ip_allocation_policy.0.services_ipv4_cidr_block
on main.tf line 45, in resource "google_composer_environment" "dwh-composer-XXXr": 45: resource "google_composer_environment" "dwh-composer-XXXr" {
Error: "config.0.node_config.0.ip_allocation_policy.0.cluster_ipv4_cidr_block": conflicts with config.0.node_config.0.ip_allocation_policy.0.cluster_secondary_range_name
on main.tf line 45, in resource "google_composer_environment" "dwh-composer-XXX": 45: resource "google_composer_environment" "dwh-composer-XXXr" {
Error: "config.0.node_config.0.ip_allocation_policy.0.services_ipv4_cidr_block": conflicts with config.0.node_config.0.ip_allocation_policy.0.services_secondary_range_name
on main.tf line 45, in resource "google_composer_environment" "dwh-composer-XXXr": 45: resource "google_composer_environment" "dwh-composer-XXXr" {
Steps to Reproduce
terraform applyImportant Factoids
References
0000