ECS: Unexpected key 'skipDestroy' found in params

[OnlyLoveOleg]

What happened?

We recently updated pulumi/awsx to 0.40.0 and after running pulumi up all the services got a new key skipDestroy in the task definition. Looks like this caused our deployment pipelines to fail with the error ECS: Unexpected key 'skipDestroy' found in params while using the GIthub action aws-actions/amazon-ecs-deploy-task-definition@v1

Steps to reproduce

Using pulumi up create service in AWS ECS.

export const fargate = new awsx.ecs.FargateService(
    `${serviceName}-fargate-${stack}`,
    {
      cluster: envEcsCluster,
      subnets: vpcPrivateSubnetIds,
      assignPublicIp: false,
      securityGroups: [ fgSg.id ],
      healthCheckGracePeriodSeconds:180,
      taskDefinitionArgs: {
        containers: {
          featuresService: {
            image: pulumi.interpolate`${registry.repository.repositoryUrl}:latest`,
            memory: 512,
            portMappings: [httpsListener],
            environment: [
              {
                name: "APP_PORT",
                value: "80",
              },
            ],
          },
        },
      },
      waitForSteadyState: false,
      desiredCount: awsEcsServiceDesiredCount,
      tags: {
        app: serviceName,
        env: stack,
      },
    }
);

Deploy using:

- name: Install pulumi
  uses: pulumi/setup-pulumi@v2
  with:
    pulumi-version: ^3.0.0
  - id: pulumi
  run: |
    pulumi login s3://${{ secrets.pulumi_aws_bucket }}
    pulumi stack select ${{ inputs.stack }}
    # read contents into CI/CD variable
    content=`pulumi stack output -j --show-secrets my_service`
    # write task_definition to json
    pulumi stack output -j --show-secrets my_service | jq .task_definition > ../${{ env.ECS_TASK_DEFINITION }}
    # the following lines are only required for multi line json
    content="${content//'%'/'%25'}"
    content="${content//$'\n'/'%0A'}"
    content="${content//$'\r'/'%0D'}"
    # end of optional handling for multi line json
    echo "::set-output name=my_service::$content"
  working-directory: ./infrastructure-v2/
  env:
    AWS_ACCESS_KEY_ID: 
    AWS_SECRET_ACCESS_KEY: 
    AWS_REGION: 
    PULUMI_CONFIG_PASSPHRASE: 

- name: Fill in the new image ID in the Amazon ECS task definition
  id: task-def
  uses: aws-actions/amazon-ecs-render-task-definition@v1
  with:
    task-definition: ${{ env.ECS_TASK_DEFINITION }}
    container-name: ${{ env.CONTAINER_NAME }}
    image: ${{ fromJson(steps.pulumi.outputs.my_service).repository_url }}:${{ github.sha }}

- name: Deploy Amazon ECS task definition
  uses: aws-actions/amazon-ecs-deploy-task-definition@v1
  with:
    task-definition: ${{ steps.task-def.outputs.task-definition }}
    service: ${{ fromJson(steps.pulumi.outputs.my_service).service }}
    cluster: ${{ fromJson(steps.pulumi.outputs.my_service).cluster }}
    wait-for-service-stability: true

Expected Behavior

New images deployed.

Actual Behavior

Failure with the message ECS: Unexpected key 'skipDestroy' found in params

Output of pulumi about

No response

Additional context

No response

Contributing

Vote on this issue by adding a 👍 reaction. To contribute a fix for this issue, leave a comment (and link to your pull request, if you've opened one already).

[roothorp]

Hi @TalLannder, thanks for the issue. Could you provide the version of pulumi-aws that you are using? It is possible that it will also need to be upgraded alongside AWSX - it looks like skipDestroy was added to pulumi-aws at the beginning of the year, with release v4.35.0

[lech-r]

@roothorp pulumi about output:

`CLI
Version 3.44.0 Go Version go1.19.2 Go Compiler gc

Plugins NAME VERSION aws 5.18.0 aws 4.37.5 docker 3.1.0 docker 3.1.0 nodejs unknown postgresql 3.3.0

Host
OS ubuntu Version 20.04 Arch x86_64

This project is written in nodejs: executable='/snap/bin/node' version='v16.18.0' `

[lech-r]

@roothorp pulumi/aws 5.18.0 pulumi/awsx 0.40.1

[lech-r]

@roothorp What feedback are you still waiting for?

[OnlyLoveOleg]

@roothorp Thank you for the prompt response. Wondering if you can help us? I see that the issue has the awaiting-response label but myself and @lech-r working together and @lech-r `s feedback is correct.

[roothorp]

Hi @lech-r & @TalLannder , sorry for the delay. We're investigating this now - apologies that I left the awaiting feedback label. I'll try and get back to you as soon as I can with a resolution.

[danielrbradley]

One theory is that it's possible that this bug is coming from a mismatch between arguments between different versions of the AWS provider - the one this was written against and the one you've installed locally ("5.18.0").

We're currently in the process of doing a fairly major overhall of these components as we move towards a stable 1.0 release. This is available in v1.0.0-beta.11 which might already solve your issue just due to how it now manages dependencies much better. It would be a good first step to know if this has already been resolved in this new implementation if you were able to test it.

[lech-r]

I don't think we'd venture into running beta in our production systems. Is there an ETA for a stable 1.0 release?

[lech-r]

@danielrbradley I see pulumi-awsx v1.0.1 is out. Should we upgrade to that and test?

[danielrbradley]

@lech-r yes, please do

[lech-r]

@danielrbradley It seems pulumi/awsx 1.0.1 introduces many breaking changes. Upgrading from 0.40.1, I am getting error TS2339: Property 'fromExistingIds' does not exist on type 'typeof Vpc' This is from awsx.ec2.Vpc.fromExistingIds call, and is just one example of the many compile problems. Until I can solve these, I won't be able to upgrade.

[lech-r]

Looks like awsx.classic may be the answer. I'll have a look at that next week.

[lech-r]

@danielrbradley I hit the wall while refactoring for the new Crosswalk. I have posted the problem to the Pulumi Slack community, but got no answer there. Could you help, please?