search

aws / aws-cdk

The AWS Cloud Development Kit is a framework for defining cloud infrastructure in code
https://aws.amazon.com/cdk
Apache License 2.0
11.67k stars 3.92k forks source link

Fix hotpatching mechanism for regression tests #8074

Closed rix0rrr closed 4 years ago

rix0rrr commented 4 years ago

The regression tests copying the "current" version of test.sh and such over into the "old" package in order to work around bugs in the old version is wrong.

Specifically, it shouldn't be copying the live version of test.sh... it should have special, versioned copies of the files it wants to hotpatch.

So in addition to

test/integ/cli/test.sh

there should be something like:

test/integ/regression/hotpatch/1.39.0/test.sh

which is the file that is copied over.

I just had the issue that I modified test.sh to depend on a new file that I added at the same time. That's all good! But because that new test.sh now also gets copied into the regression tests, it has a dependency on that new file, but that new file isn't copied yet!

The fact that at some point in time, the "live" test.sh and the backported/bugfixed "test.sh" are the same file, doesn't mean they will always be the same file... so they shouldn't have been the same file!

Also if we version it, the hotpatching will naturally age out as it becomes irrelevant, instead of us having to remember to go back and remove the patching

This is :bug: Bug Report

iliapolo commented 4 years ago

@rix0rrr Is this still relevant? I seem to remember a PR you made that addressed this...no?

iliapolo commented 4 years ago

Closing as this looks to have been fixed by https://github.com/aws/aws-cdk/pull/8403/files