pulumi / providertest

Incubating facilities for testing Pulumi providers
Apache License 2.0
5 stars 0 forks source link

Retain temporary folders for inspection when tests failed #81

Closed t0yv0 closed 6 months ago

t0yv0 commented 6 months ago

This came out of the friction of debugging a failed upgrade test. ProgramTest had a nice feature that left a folder in TMP with the current state right at the point of failure that can be further inspected in the CLI, printing the path to the folder when that happens.

This change moves us closer to the desired experience by substituting t.TempDir() with an inlined version of TempDir that does not delete it on failure; and similarly it stops destroying stacks for failed tests.

There is some further ergonomics to work out such as emitting an .envrc file so that all the env vars get picked up in the dir and the user does not need to chase down the file backend URL, passphrase and so on.

codecov[bot] commented 6 months ago

Codecov Report

Attention: Patch coverage is 64.83516% with 32 lines in your changes are missing coverage. Please review.

Project coverage is 37.99%. Comparing base (3a38438) to head (2102eca). Report is 2 commits behind head on main.

:exclamation: Current head 2102eca differs from pull request most recent head b1a11a9. Consider uploading reports for the commit b1a11a9 to get more accurate results

Files Patch % Lines
pulumitest/tempdir.go 73.43% 12 Missing and 5 partials :warning:
pulumitest/testingT.go 30.76% 8 Missing and 1 partial :warning:
pulumitest/newStack.go 50.00% 4 Missing and 1 partial :warning:
pulumitest/ciutil.go 66.66% 0 Missing and 1 partial :warning:
Additional details and impacted files ```diff @@ Coverage Diff @@ ## main #81 +/- ## ========================================== + Coverage 37.35% 37.99% +0.64% ========================================== Files 43 46 +3 Lines 2677 2779 +102 ========================================== + Hits 1000 1056 +56 - Misses 1569 1607 +38 - Partials 108 116 +8 ```

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.

VenelinMartinov commented 6 months ago

Did you mean to push changes here? I don't see any change since the last review

t0yv0 commented 6 months ago

Pushed changes to address feedback .Thanks!