search

hashicorp / terraform-provider-aws

The AWS Provider enables Terraform to manage AWS resources.
https://registry.terraform.io/providers/hashicorp/aws
Mozilla Public License 2.0
9.81k stars 9.16k forks source link

Resource 'aws_api_gateway_deployment' in 2.3.0 does not have (optional) attribute 'stage_name' #8049

Closed clocked0ne closed 3 years ago

clocked0ne commented 5 years ago

Community Note

Terraform Version

Affected Resource(s)

Terraform Configuration Files

resource "aws_api_gateway_rest_api" "app" {
  name        = "${var.name}"
  description = "redacted"
}

resource "aws_api_gateway_api_key" "app" {
  name = "${var.name}"
}

resource "aws_api_gateway_usage_plan" "app" {
  name        = "${var.name}"
  description = "Unlimited usage plan"

  api_stages {
    api_id = "${aws_api_gateway_rest_api.app.id}"
    stage  = "${var.stage_name}"
  }

  depends_on = ["aws_api_gateway_stage.app"]
}

resource "aws_api_gateway_usage_plan_key" "app" {
  key_id        = "${aws_api_gateway_api_key.app.id}"
  key_type      = "API_KEY"
  usage_plan_id = "${aws_api_gateway_usage_plan.app.id}"
}

resource "aws_api_gateway_deployment" "app" {
  stage_description = "${var.gateway_version}"
  # stage_name        = "${var.stage_name}"
  description       = "${var.gateway_version}"
  rest_api_id       = "${aws_api_gateway_rest_api.app.id}"
  depends_on        = ["aws_api_gateway_integration.app"]
}

resource "aws_api_gateway_stage" "app" {
  stage_name    = "${var.stage_name}"
  rest_api_id   = "${aws_api_gateway_rest_api.app.id}"
  deployment_id = "${aws_api_gateway_deployment.app.id}"
}

Debug Output

https://gist.github.com/clocked0ne/f3932eea73bec35b289ac952035c6f12

Panic Output

Expected Behavior

Terraform Apply should have completed successfully as the stage_name in aws_api_gateway_deployment is now optional

Actual Behavior

The Terraform Apply failed with the error described.

Steps to Reproduce

  1. terraform apply

Important Factoids

In the issue referenced below @bflad suggested:

if I was to offer a guess without seeing your configuration and plan output, you cannot currently reference the stage_name as an attribute if it is not defined as an argument or specified as an empty string (e.g. stage_name = ""). Support for this will require an additional lookup of the stage name based on the deployment ID, which the resource should have been doing before anyways for drift detection.

Or put more thoroughly for someone who might want to fix this, we need to get acceptance tests passing with resource.TestCheckResourceAttr() (if the value is known) or resource.TestCheckResourceAttrSet() (if the value is unknown) instead of resource.TestCheckNoResourceAttr() in the acceptance tests here:

terraform-provider-aws/aws/resource_aws_api_gateway_deployment_test.go

Lines 125 to 171 in b4d6106 func TestAccAWSAPIGatewayDeployment_StageName(t *testing.T) { var deployment apigateway.Deployment var stage apigateway.Stage resourceName := "aws_api_gateway_deployment.test"

resource.ParallelTest(t, resource.TestCase{ PreCheck: func() { testAccPreCheck(t) }, Providers: testAccProviders, CheckDestroy: testAccCheckAWSAPIGatewayDeploymentDestroy, Steps: []resource.TestStep{ { Config: testAccAWSAPIGatewayDeploymentConfigStageName("test"), Check: resource.ComposeTestCheckFunc( testAccCheckAWSAPIGatewayDeploymentExists(resourceName, &deployment), testAccCheckAWSAPIGatewayDeploymentStageExists(resourceName, &stage), resource.TestCheckResourceAttr(resourceName, "stage_name", "test"), ), }, { Config: testAccAWSAPIGatewayDeploymentConfigRequired(), Check: resource.ComposeTestCheckFunc( resource.TestCheckNoResourceAttr(resourceName, "stage_name"), ), }, }, }) }

func TestAccAWSAPIGatewayDeployment_StageName_EmptyString(t *testing.T) { var deployment apigateway.Deployment resourceName := "aws_api_gateway_deployment.test"

resource.ParallelTest(t, resource.TestCase{ PreCheck: func() { testAccPreCheck(t) }, Providers: testAccProviders, CheckDestroy: testAccCheckAWSAPIGatewayDeploymentDestroy, Steps: []resource.TestStep{ { Config: testAccAWSAPIGatewayDeploymentConfigStageName(""), Check: resource.ComposeTestCheckFunc( testAccCheckAWSAPIGatewayDeploymentExists(resourceName, &deployment), resource.TestCheckNoResourceAttr(resourceName, "stage_name"), ), }, }, }) }

Can you please open a new GitHub issue filling out all the details so we can ensure we can appropriately triage your situation? Thanks.

References

sousmangoosta commented 5 years ago

Hi @clocked0ne ,

I tried a run like your who works like a charm.

Terraform Configuration

provider "aws" {
  region = "${var.region}"
  version = "2.3.0"
}

variable "name" {
  default = "api-name"
}

variable "stage_name" {
  default = "v1"
}

variable "gateway_version" {
  default = "test API"
}

resource "aws_api_gateway_rest_api" "app" {
  name        = "${var.name}"
  description = "redacted"
}

resource "aws_api_gateway_api_key" "app" {
  name = "${var.name}"
}

resource "aws_api_gateway_usage_plan" "app" {
  name        = "${var.name}"
  description = "Unlimited usage plan"

  api_stages {
    api_id = "${aws_api_gateway_rest_api.app.id}"
    stage  = "${var.stage_name}"
  }

  depends_on = ["aws_api_gateway_stage.app"]
}

resource "aws_api_gateway_usage_plan_key" "app" {
  key_id        = "${aws_api_gateway_api_key.app.id}"
  key_type      = "API_KEY"
  usage_plan_id = "${aws_api_gateway_usage_plan.app.id}"
}

resource "aws_api_gateway_deployment" "app" {
  stage_description = "${var.gateway_version}"
  description       = "${var.gateway_version}"
  rest_api_id       = "${aws_api_gateway_rest_api.app.id}"
  depends_on        = ["aws_api_gateway_integration.app"]
}

resource "aws_api_gateway_stage" "app" {
  stage_name    = "${var.stage_name}"
  rest_api_id   = "${aws_api_gateway_rest_api.app.id}"
  deployment_id = "${aws_api_gateway_deployment.app.id}"
}

resource "aws_api_gateway_resource" "app" {
  rest_api_id = "${aws_api_gateway_rest_api.app.id}"
  parent_id   = "${aws_api_gateway_rest_api.app.root_resource_id}"
  path_part   = "mydemoresource"
}

resource "aws_api_gateway_method" "app" {
  rest_api_id   = "${aws_api_gateway_rest_api.app.id}"
  resource_id   = "${aws_api_gateway_resource.app.id}"
  http_method   = "POST"
  authorization = "NONE"
}

resource "aws_api_gateway_integration" "app" {
  http_method = "${aws_api_gateway_method.app.http_method}"
  resource_id = "${aws_api_gateway_resource.app.id}"
  rest_api_id = "${aws_api_gateway_rest_api.app.id}"
  type = "MOCK"
}

Output

https://gist.github.com/sousmangoosta/4b5f2511e9357ee59d79cdab1d5c0f97

clocked0ne commented 5 years ago

Strange, I can only think that it is either because there is a race condition or it was not a 'fresh' deployment when we ran it as some elements of the config had changed. It would be good for someone else to be able to verify this with their own example.

kgrodzicki commented 5 years ago

@clocked0ne I can confirm, issue is still in version 2.3.0, getting error: Resource 'aws_api_gateway_deployment.test' does not have attribute 'stage_name' for variable 'aws_api_gateway_deployment.test.stage_name' when 'stage_name' is missing in 'aws_api_gateway_deployment'

sousmangoosta commented 5 years ago

@clocked0ne @kgrodzicki : Can you provide a usable as-is example of non working terraform configuration, could you please give a real debug output with command like TF_LOG=DEBUG terraform apply

kgrodzicki commented 5 years ago

Cannot reproduce it with version 2.4.0. LGTM 👍

WilsonSo commented 5 years ago

Does this mean this is resolved in 2.4.0 aws provider and this issue can therefore be closed?

clocked0ne commented 5 years ago

@sousmangoosta I am unable to test with a fresh configuration at this point but assume based on the responses from @kgrodzicki that the problem is fixed, I have seen no further issues.

DmitryRendov commented 5 years ago

I'm still having this issue using 2.12.0 version. Planning runs fine in case of stage_name="", when apply I'm getting "Error creating API Gateway Stage: BadRequestException: Stage name must be non-empty"

upd: I can see that this option is still required - https://github.com/terraform-providers/terraform-provider-aws/blame/master/aws/resource_aws_api_gateway_stage.go#L99

sousmangoosta commented 5 years ago

I'm still having this issue using 2.12.0 version. Planning runs fine in case of stage_name="", when apply I'm getting "Error creating API Gateway Stage: BadRequestException: Stage name must be non-empty"

upd: I can see that this option is still required - https://github.com/terraform-providers/terraform-provider-aws/blame/master/aws/resource_aws_api_gateway_stage.go#L99

Yes for resource "aws_api_gateway_stage"the stage_name is required, it's not required for resource "aws_api_gateway_deployment"

SemiConscious commented 4 years ago

@sousmangoosta sorry to come back to this after such a long time, but it's still an issue. The problem with your test script from 23rd March 2019 is that aws_api_gateway_deployment.app doesn't have the stage_name set, just stage_description. This will deploy just fine, but the API won't be accessible. The problem everyone is having is where you need to have aws_api_gateway_stage and aws_api_gateway_deployment with both having the same stage name set. In my case, deploying from scratch results in a 'stage already exists' error when creating the aws_api_gateway_stage. Manually deleting the stage and redeploying works, but it's too hand-held to be a solution.

A potential solution is presented in 'important factoids' at the top of this ticket.

My particular use case is: enable api gateway access logging. It's not possible to enable this without a aws_api_gateway_stage, and it's not possible to deploy a working api using a aws_api_gateway_stage as things stand.

shederman commented 4 years ago

I have been reading through the trail of tickets related to this (from March 2019!) with incorrect and misleading information, closed after inactivity, and incorrect solutions. This is a critical problem which makes Terraform not fit for purpose for any use of API Gateway that:

The suggested solution in important factoids only works if you are creating all the API Gateway Resources by hand rather than using body and importing the Swagger.

In other words, it means anyone using API Gateway in a non-toy project essentially must avoid Terraform as a deployment mechanism or accept a manual deployment step.

sousmangoosta commented 4 years ago

@SemiConscious I upgraded my example here :

provider "aws" {
  region = var.region
  version = "2.65.0"
}

variable "name" {
  default = "api-name"
}

variable "stage_name" {
  default = "v1"
}

variable "gateway_version" {
  default = "test API"
}

resource "aws_api_gateway_rest_api" "app" {
  name        = var.name
  description = "redacted"
}

resource "aws_api_gateway_api_key" "app" {
  name = var.name
}

resource "aws_api_gateway_usage_plan" "app" {
  name        = var.name
  description = "Unlimited usage plan"

  api_stages {
    api_id = aws_api_gateway_rest_api.app.id
    stage  = var.stage_name
  }

  depends_on = [aws_api_gateway_stage.app]
}

resource "aws_api_gateway_usage_plan_key" "app" {
  key_id        = aws_api_gateway_api_key.app.id
  key_type      = "API_KEY"
  usage_plan_id = aws_api_gateway_usage_plan.app.id
}

resource "aws_api_gateway_deployment" "app" {
  stage_description = var.gateway_version
  description       = var.gateway_version
  rest_api_id       = aws_api_gateway_rest_api.app.id
  depends_on        = [aws_api_gateway_integration.app]
}

resource "aws_api_gateway_stage" "app" {
  stage_name    = var.stage_name
  rest_api_id   = aws_api_gateway_rest_api.app.id
  deployment_id = aws_api_gateway_deployment.app.id
}

resource "aws_api_gateway_resource" "app" {
  rest_api_id = aws_api_gateway_rest_api.app.id
  parent_id   = aws_api_gateway_rest_api.app.root_resource_id
  path_part   = "mydemoresource"
}

resource "aws_api_gateway_method" "app" {
  rest_api_id   = aws_api_gateway_rest_api.app.id
  resource_id   = aws_api_gateway_resource.app.id
  http_method   = "POST"
  authorization = "NONE"
}

resource "aws_api_gateway_integration" "app" {
  http_method = aws_api_gateway_method.app.http_method
  resource_id = aws_api_gateway_resource.app.id
  rest_api_id = aws_api_gateway_rest_api.app.id
  type = "MOCK"
}

resource "aws_api_gateway_method_response" "app" {
  rest_api_id = aws_api_gateway_rest_api.app.id
  resource_id = aws_api_gateway_resource.app.id
  http_method = aws_api_gateway_method.app.http_method
  status_code = "200"
}

resource "aws_api_gateway_integration_response" "MyDemoIntegrationResponse" {
  rest_api_id = aws_api_gateway_rest_api.app.id
  resource_id = aws_api_gateway_resource.app.id
  http_method = aws_api_gateway_method.app.http_method
  status_code = aws_api_gateway_method_response.app.status_code
}

output "api_url" {
  value = [aws_api_gateway_stage.app.invoke_url,aws_api_gateway_resource.app.path_part]
}

My version :

$ terraform version
Terraform v0.12.26
$

Output

https://gist.github.com/sousmangoosta/4b5f2511e9357ee59d79cdab1d5c0f97

API call

https://gist.github.com/sousmangoosta/339aa5a0ea3731a1eda75576ce0a9020

sousmangoosta commented 4 years ago

@shederman Could you please provide an example ?

davidkarlsen commented 4 years ago

I don't see a WAF option for any of the resources?

sousmangoosta commented 4 years ago

@davidkarlsen You got an example of associate an Waf ACL to API Gateway here on the last block aws_wafv2_web_acl_association

bflad commented 3 years ago

Hi folks 👋

The stage_name attribute of the aws_api_gateway_deployment resource should be avoided where possible in configurations (both as an argument or an attribute reference) and it will likely be deprecated soon. It serves a confusing purpose of managing a stage during deployment creation that typically does not work well with Terraform's design. My initial comment from the previous issue suggesting that the aws_api_gateway_deployment resource could theoretically lookup a stage name would only work if there was one stage associated with a deployment, however since multiple stages can be associated with a deployment attempting to fill it in would result inaccurate results for the value.

In this situation, it is better to use the aws_api_gateway_stage resource stage_name attribute instead of aws_api_gateway_deployment resource stage_name argument.

To summarize this in a quick configuration example:

resource "aws_api_gateway_deployment" "example" {
  rest_api_id = aws_api_gateway_rest_api.example.id

  triggers = {
    redeployment = # ... references to upstream resources (see issue link below) ...
  }

  lifecycle {
    create_before_destroy = true
  }
}

resource "aws_api_gateway_stage" "example" {
  deployment_id = aws_api_gateway_deployment.example.id
  rest_api_id   = aws_api_gateway_rest_api.example.id
  stage_name    = "example"
}

# ... downstream references should be to aws_api_gateway_stage.example.stage_name ...

If you are having further trouble with this functionality after applying these recommendations, please submit new GitHub issues following the bug report and feature request issue templates for further triage.

ghost commented 3 years ago

I'm going to lock this issue because it has been closed for 30 days ⏳. This helps our maintainers find and focus on the active issues.

If you feel this issue should be reopened, we encourage creating a new issue linking back to this one for added context. Thanks!