Open t0yv0 opened 1 year ago
CC @danielrbradley
Do we really need credentials for these kinds of tests? I've been using a script I cribbed from one of Ian's reviews to force my tests to use local state.
export PULUMI_CONFIG_PASSPHRASE=""
mkdir -p state
pulumi login --cloud-url "file://$PWD/state"
pulumi stack select organization/test/test --create
# pulumi up or whatever here
pulumi stack rm --yes
pulumi logout
rm -Rf "$PWD/state"
Though perhaps that's all you meant by setting up credentials
Oh, I'm realizing now that you mean setting up the GCP credentials. I'm still confused why we need credentials for GCP to run what are essentially local only tests
It's very unfortunate that creds are required for these tests. But the provider does a creds check too eagerly. We could maybe figure out a way to skip creds check in a test context.
Quick-running tests that require credentials need a place to run in CI for bridged providers.
This blocks https://github.com/pulumi/pulumi-gcp/pull/1264
Recently ci-mgmt introduced a change to run unit tests on the
./provider
module as part ofprerequisites
. This is a great place to run tests of this form because the Go build caches are hot and these tests are fast to run, if they fail they block off the more significant test matrix jobs.There are tests that are not quite unit but not full integration either.
Unlike unit tests, these tests do not create resources or touch the cloud, but they might exercise
pulumi preview
or the real provider in a way that makes it fail unless credentials are supplied.Unlike integration tests, they still complete within <1 minute(s)
The concrete proposal here is to run these tests as part of
prerequisites
job in CI also, by editing this job to establish cloud credentials for the runner.