aws-cloudformation / cloudformation-resource-schema

The CloudFormation Resource Schema defines the shape and semantic for resources provisioned by CloudFormation. It is used by provider developers using the CloudFormation RPDK.
Apache License 2.0
90 stars 38 forks source link

PublicTypeVersion NOT actually incrementing public version (?) #130

Closed SHxKM closed 2 years ago

SHxKM commented 2 years ago

Following up on the thread where I reported an issue (#129) which actually turned out to be an issue on the CFN side, I used a workaround suggested there by @ugudip for publishing a public type:

Hey SHxKM, Thanks for the template. I don't think it makes any difference by changing the resource name you give, you are allowed to give any name as a logical Id. Your template looks good at first glance, but I will need to investigate on why it isn't working for you. I will get back to you shortly with my findings. Meanwhile, as a workaround, can you please try by giving TypeName and Type instead of Arn for PublicTypeVersion resource?

So I've used this template to register the custom resource in almost all regions:

Resources:
  MyTypePrivateTypeVersion: 
    Type: AWS::CloudFormation::ResourceVersion
    Properties:
      SchemaHandlerPackage: !Ref SchemaHandlerUrl
      TypeName: MyCompany::MyCategory::MyType
  MyTypeResourceDefaultVersion:    
    Type: AWS::CloudFormation::ResourceDefaultVersion
    DependsOn: MyTypePrivateTypeVersion
    Properties:
      TypeVersionArn: !Ref MyTypePrivateTypeVersion
  CustomTypePublisher:
    Type: AWS::CloudFormation::Publisher
    DependsOn: MyTypeResourceDefaultVersion
    Properties:
      AcceptTermsAndConditions: true
  MyTypePublicTypeVersion:     
    Type: AWS::CloudFormation::PublicTypeVersion
    DependsOn: CustomTypePublisher
    Properties:
      LogDeliveryBucket: my-s3-logs-bucket
      PublicVersionNumber: !Ref VersionToPublish
      Type: RESOURCE
      TypeName: MyCompany::MyCategory::MyType

The registration of version 1.0.0 was successful.

However, when I tried to update the version to 1.0.1, even though the operation was "successful", the version isn't really bumped.

Despite all this, almost 40 minutes after the "successful" publishing, the version showing in the public registry is still 1.0.0. It seems that there is no real sync whatsoever.

There is one detail that may be affecting this process. When I registered the types initially, I used:

aws cloudformation create-stack-instances --stack-set-name my-stack-set-name --accounts '["XXXXX"]'  --regions '["LIST OF REGIONS HERE"]' --operation-preferences FailureToleranceCount=0,MaxConcurrentCount=1,RegionConcurrencyType=PARALLEL

The StackSet parameters were:

VersionToPublish=1.0.1 # this may be the culprit 
SchemaHandlerUrl=s3://my-bucket/my-handler.zip

Notice the 1.0.1 as the parameter value. What this did: it published the public types successfully, but as version 1.0.0 - this is documented in PublicTypeVersion:

The first time you publish a type, AWS CloudFormation sets the version number to 1.0.0, regardless of the value you specify.

Any attempts to then update the template failed, or reported "No updates are to be performed" (because the template showed 1.0.1, even though the actual underlying resource version was still 1.0.0, so we were stuck. As a workaround, I've uploaded an identical .zip to my-handler.zip, and updated the stack-set changing only the handler URL to the new (yet identical zip):

aws cloudformation update-stack-set --stack-set-name my-stack-set-name --template-body "file://my-public-type-stackset.yaml" --region us-east-1 --parameters ParameterKey=VersionToPublish,ParameterValue=1.0.1 ParameterKey=SchemaHandlerUrl,ParameterValue=s3://my-bucket/my-handler-updated.zip

This operation was successful, but as I detailed above, it does not seem to actually publish the new version. What is the proper way to "force" all stack instances to update, given the same parameters?

SHxKM commented 2 years ago

Even expanding the dependencies for MyTypePublicTypeVersion and changing the zip handler once again, results in the same outcome: The update operation is successful, no new version is actually published.

  MyTypePublicTypeVersion:     
    Type: AWS::CloudFormation::PublicTypeVersion
    DependsOn: 
      - CustomTypePublisher
      - MyTypePrivateTypeVersion
      - MyTypeResourceDefaultVersion
    Properties:
      LogDeliveryBucket: my-s3-logs-bucket
      PublicVersionNumber: !Ref VersionToPublish
      Type: RESOURCE
      TypeName: MyCompany::MyCategory::MyType

If I try to bump to 1.0.2 as a sort of a brute-force method, I get an error that patch versions must be incremented by 1, which means it has to be 1.0.1.

SHxKM commented 2 years ago

The only "solution" seems to be deleting the stack instance(s), and recreating it after the public type has already been registered (it won't get deleted when the stack instance is deleted). But this is:

  1. Very tedious
  2. Unnecessarily dangerous in a production environment and a sensitive AWS account

There should be a way to detect version differences, or AWS::CloudFormation::PublicTypeVersion should fail if upon a first time publish, the version specified is >1.0.0.

mayankta commented 2 years ago

@SHxKM we are investigating the issue, and will get back with the findings

mayankta commented 2 years ago

@SHxKM so PublicVersionNumber was passed in the template even for first time publishing of the resource, this caused the issue, we will update the docs to reflect when we should pass the PublicVersionNumber also we will made a change to throw validation error to handle this scenario so that stack won't be created. Keeping the issue open until fix is pushed.

SHxKM commented 2 years ago

so PublicVersionNumber was passed in the template even for first time publishing of the resource

Yes, more precisely a version number > 1.0.0.

also we will made a change to throw validation error to handle this scenario so that stack won't be created. Keeping the issue open until fix is pushed.

Thank you

mayankta commented 2 years ago

Hi @SHxKM Fix is now deployed. You should see a validation error now if you are publishing for first time and try to create a stack with PublicVersionNumber . Closing this issue now.