search

aws / aws-cdk

The AWS Cloud Development Kit is a framework for defining cloud infrastructure in code
https://aws.amazon.com/cdk
Apache License 2.0
11.66k stars 3.92k forks source link

Inevitable deployment breaking library upgrades #3422

Closed andrey-chorniy closed 5 years ago

andrey-chorniy commented 5 years ago

We were using aws-cdk typescript version since version 0.22.0. We did a couple of updates up to the 0.34.0 and we able to deploy our stacks. We have migrated code from 0.34.0 to 0.36.1 and tried to deploy. On of our stacks - DynamoDbStack (it deployed DynamoDB resourced) appeared in status UPDATE_ROLLBACK_COMPLETE. The problem which happened was

Attribute: StreamArn was not found for resource: ehawk-mobilehub-1778525198-Events

The organization of the stacks was as follows:

  1. DynamoDbStack:
    • defined dynamodb tables with stream enabled
  2. LambdaStack
    • defined lambas part of which were set up as triggers for DynamoDB tables

The fatal problem which this UPDATE_ROLLBACK_COMPLETE introduced was the total removal of lambda triggers that were present on DynamoDB tables. Of course it immediately broke the app, but what's more important, there was no way to roll it back fast enough (it took a couple of hours and manual actions with the stacks from our side)

Release log of 0.36.0 as well as 0.35.0 had a lot of declared "Breaking changes" which are probably the reason of this issue. But it's really unclear what we have to do and if it is even possible to upgrade to that version from 0.34.0 without dropping the existing environment.

Q1 Is there some documentation we can use to find out how to perform upgrades from one version to another?

Q2 Can we expect changes in the SDK that under the certain organization of CloudFormation stacks will inevitably and with no way for automatic rollback break the deployment after code upgrade like happened in our case?

Q3 Does the "1.x" version means that now aws-cdk project could be considered as "Stable" and we may expect some level of stability and guarantees that we can perform "1.x" upgrades without major issues as we described?

eladb commented 5 years ago

I am sorry to hear about this really lousy experience. I hope that this didn't cause too much grief. Pre 1.x versions of the CDK were Developer Preview and as advertised they were susceptible to breaking changes, like the ones you described.

1.x onwards is generally a stable (GA) version and APIs are expected not to break, unless a module or API is marked as "experimental". It should be visible in docs and in your IDE.

I am closing this for now. Reopen if you feel there's something we can help with.