Service Catalog product version changes doesn't create a new version

will-schneble commented 3 years ago

Community Note

Please vote on this issue by adding a 👍 reaction to the original issue to help the community and maintainers prioritize this request
Please do not leave "+1" or other comments that do not add relevant new information or questions, they generate extra noise for issue followers and do not help prioritize the request
If you are interested in working on this issue or have submitted a pull request, please leave a comment

Terraform CLI and Terraform AWS Provider Version

Terraform v0.15.3
on windows_amd64
+ provider registry.terraform.io/hashicorp/aws v3.42.0

Affected Resource(s)

aws_servicecatalog_product

Terraform Configuration Files

terraform {
  backend "local" {}
  required_providers {
    aws = {
      source  = "hashicorp/aws"
      version = ">= 3.42.0"
    }
  }
}

provider "aws" {
  region = "us-east-1"
}

data "aws_caller_identity" "this" {}

resource "aws_servicecatalog_product" "this" {
  name        = "example-product"
  owner       = data.aws_caller_identity.this.account_id
  type        = "CLOUD_FORMATION_TEMPLATE"
  description = "Template description."

  provisioning_artifact_parameters {
    template_url = "https://s3.amazonaws.com/bucket/template.yaml" #replace this with a valid link
    name         = "0.0.1"
    description  = "init"
    type         = "CLOUD_FORMATION_TEMPLATE"
  }
}

Debug Output

Terraform used the selected providers to generate the following execution plan. Resource actions are indicated with the following symbols:
  + create

Terraform will perform the following actions:

  # aws_servicecatalog_product.this will be created
  + resource "aws_servicecatalog_product" "this" {
      + accept_language      = "en"
      + arn                            = (known after apply)
      + created_time            = (known after apply)
      + description               = "Template description."
      + distributor                = (known after apply)
      + has_default_path      = (known after apply)
      + id                              = (known after apply)
      + name                        = "example-product"
      + owner                       = "123456789123"
      + status                       = (known after apply)
      + support_description = (known after apply)
      + support_email          = (known after apply)
      + support_url              = (known after apply)
      + tags_all                    = (known after apply)
      + type                         = "CLOUD_FORMATION_TEMPLATE"

      + provisioning_artifact_parameters {
          + description                  = "init"
          + disable_template_validation = false
          + name                           = "0.0.1"
          + template_url                = "https://s3.amazonaws.com/bucket/template.yaml"
          + type                             = "CLOUD_FORMATION_TEMPLATE"
        }
    }

Plan: 1 to add, 0 to change, 0 to destroy.

Do you want to perform these actions?
  Terraform will perform the actions described above.
  Only 'yes' will be accepted to approve.

  Enter a value: yes

aws_servicecatalog_product.this: Creating...
aws_servicecatalog_product.this: Creation complete after 2s [id=prod-it2w7uijwepek]

Apply complete! Resources: 1 added, 0 changed, 0 destroyed.

Then changing the provisioning_artifact_parameters attributes:

aws_servicecatalog_product.this: Refreshing state... [id=prod-it2w7uijwepek]

Terraform used the selected providers to generate the following execution plan. Resource actions are indicated with the following symbols:
  ~ update in-place

Terraform will perform the following actions:

  # aws_servicecatalog_product.this will be updated in-place
  ~ resource "aws_servicecatalog_product" "this" {
        id                   = "prod-it2w7uijwepek"
        name             = "example-product"
        tags               = {}
        # (9 unchanged attributes hidden)

      ~ provisioning_artifact_parameters {
          ~ description                 = "init" -> "minor release"
          ~ name                          = "0.0.1" -> "0.1.0"
            # (3 unchanged attributes hidden)
        }
    }

Plan: 0 to add, 1 to change, 0 to destroy.

Do you want to perform these actions?
  Terraform will perform the actions described above.
  Only 'yes' will be accepted to approve.

  Enter a value: yes

aws_servicecatalog_product.this: Modifying... [id=prod-it2w7uijwepek]
aws_servicecatalog_product.this: Modifications complete after 0s [id=prod-it2w7uijwepek]

Apply complete! Resources: 0 added, 1 changed, 0 destroyed.

Panic Output

Expected Behavior

Expected a new product version in the second apply with name "0.1.0" and description "minor release".

Actual Behavior

Nothing is created in AWS Service Catalog. The Terraform state file shows the changes but they are not reflected in the AWS console.

Steps to Reproduce

terraform apply
Change the name, description, or template_url of provisioning_artifact_parameters block.
terraform apply

Important Factoids

The initial product version will also not be recreated if it is deleted which can be done by doing the following: 1.terraform apply

In the AWS console, create a new product version manually.
Delete the original product version.
terraform apply and no changes are made.

References

ewbankkit commented 3 years ago

It looks like we are missing ForceNew: true, on each of the arguments in the provisioning_artifact_parameters configuration block:

https://github.com/hashicorp/terraform-provider-aws/blob/5283e38ff9fa019c0d8aa3059a41a03cf431a2c2/aws/resource_aws_servicecatalog_product.go#L68-L111

ewbankkit commented 3 years ago

19408

will-schneble commented 3 years ago

Workaround exists for provider >=3.43.0 with the aws_servicecatalog_provisioning_artifact https://github.com/hashicorp/terraform-provider-aws/pull/19316 .

But this does mean a dummy/initial CFT must be uploaded and active with the aws_servicecatalog_product resource because the provisioning_artifact_parameters is required and doesn't have an artifact enable/disable parameter.

markvl commented 2 years ago

We've got a similar problem which (I think) requires the same solution: adding ForceNew: true on each of the arguments.

In our case we create the template via an aws_s3_bucket_object resource using a templatefile. When that template is updated, and as a result the bucket object, we would like to update the product so the new template is used in an active version.

resource "aws_s3_bucket_object" "template" {
  key      = "product.template"
  bucket   = aws_s3_bucket.my_templates.id
  content  = templatefile("${path.module}/templates/product.yml", {})
}

resource "aws_servicecatalog_product" "this" {
  name     = "example"
  owner    = data.aws_caller_identity.this.account_id
  type     = "CLOUD_FORMATION_TEMPLATE"

  provisioning_artifact_parameters {
    type                        = "CLOUD_FORMATION_TEMPLATE"
    template_url                = "https://${aws_s3_bucket.my_templates.bucket_domain_name}/${aws_s3_bucket_object.template.key}?version_id=${aws_s3_bucket_object.template.version_id}"
  }
}

What happens is that even though the template (and with it the template_url attribute) is changed, the product version unfortunately remains unchanged and still uses the old template.

For now we use an additional, separate aws_servicecatalog_provisioning_artifact as a workaround since this resource is replaced when the template is updated. But that leaves us with the initial version that was created via the provisioning_artifact_parameters and cannot be deactivated via Terraform. (Basically the same as what @will-schneble mentioned in the previous comment.)

mailjunze commented 2 years ago

@markvl Facing the same issue. I used a placeholder version while creating the product .

provisioning_artifact_parameters {
    name= "-"
    type  = "CLOUD_FORMATION_TEMPLATE"
    description ="placeholder version-DO NOT launch"
    template_url = "https://${aws_s3_bucket.my_templates.bucket_domain_name}/${aws_s3_bucket_object.template.key}?version_id=${aws_s3_bucket_object.template.version_id}"
  }

placeholder_template.yaml

'AWSTemplateFormatVersion': '2010-09-09'
'Conditions':
  'ShouldDoAnything':
    'Fn::Equals':
    - !!bool 'true'
    - !!bool 'false'
'Resources':
  'NoOp':
    'Type': 'AWS::S3::Bucket'
    'Condition': 'ShouldDoAnything'
'Rules': {}

However, I am still concerned how do update same version with the new cfn template without changing the file name ? How can I deploy/update the version with the latest changes to the template keeping the file name unchanged ? Changing the aws_servicecatalog_provisioning_artifact.name wont pick up the latest template. :(

markvl commented 2 years ago

@mailjunze My workaround looks like this:

resource "aws_s3_bucket_object" "template" {
    key      = "product.template"
    bucket   = aws_s3_bucket.my_templates.id
    content  = templatefile("${path.module}/templates/product.yml", {})
  }

resource "aws_servicecatalog_product" "this" {
  name     = "example"
  owner    = data.aws_caller_identity.current.id
  type     = "CLOUD_FORMATION_TEMPLATE"

  # Dummy version.
  # Due to https://github.com/hashicorp/terraform-provider-aws/issues/19506 we
  # have to use a separate resource here so the product version will be updated
  # when the template is changed.
  provisioning_artifact_parameters {
    type                        = "CLOUD_FORMATION_TEMPLATE"
    name                        = "don't use"
    disable_template_validation = true
    template_url                = "invalid"
  }
}

# Will be redundant once https://github.com/hashicorp/terraform-provider-aws/issues/19506 is fixed.
resource "aws_servicecatalog_provisioning_artifact" "this" {
  name         = "latest"
  type         = "CLOUD_FORMATION_TEMPLATE"
  template_url = "https://${aws_s3_bucket.my_templates.bucket_domain_name}/${aws_s3_bucket_object.template.key}?version_id=${aws_s3_bucket_object.template.version_id}"
  product_id   = aws_servicecatalog_product.this.id
}

When there is a change in the template, the aws_servicecatalog_provisioning_artifact is also replaced and thus the version called "latest" is up to date with the template in S3.

tonyszhang commented 2 years ago

Another way that also works well in case you use a null_resource to sync artifacts to S3 (instead of aws_s3_bucket_object), is to go with the documented A "provisioning artifact" is also referred to as a "version.". So that you manually update resource "aws_servicecatalog_provisioning_artifact" "v1.0.0" {} to get a deploy.

TheDataPlatform commented 1 year ago

Ran into the same thing. The issue being that we need to trigger a change. Terraform knows my yaml file has been updated, it just never fully overwrites the service catalog product. As others have mentioned, updating the version never makes its way to the catalog. The things we're missing is "ForceNew". There's another way we can do that.

Hash your file and add for_each:

locals {
    my_hash = filesha1("<path-to-your-file>")
}

resource "aws_servicecatalog_product" "cluster-template" {
  for_each = toset([local.my_hash])
  ...
}

While a bit clunky, hashing the file gives us a unique value each time our file has been updated. Adding for_each to our resource would essentially "ForceNew" for_each hash in our list. So when our file updates, our hash updates, and we get a new resource.

hashicorp / terraform-provider-aws