This repo contains the upgrade-provider
tool. upgrade-provider
aims to reduce the
amount of human intervention necessary for upgrading bridged Pulumi providers to a limit of zero.
With our current bridged provider structure, we can reduce provider upgrades to 3 manual operations:
resources.go
, if using (we recommend upgrading to auto token mapping!) // TODO: send a link to exampleThe rest of an upgrade is formulaic, and can thus be automated. This tool attempts to do the rest.
go install github.com/pulumi/upgrade-provider@main
1.20
git
version >=2.36.0
Additionally, upgrade-provider
relies on all tools necessary for a manual provider upgrade.
That generally means pulumi
, make
, and the build toolchain for each released SDK.
To take full advantage of this tool, you can choose to use the new auto token mapping functionality. This feature is still experimental, and may be subject to change. An implementation example can be found in pulumi-okta.
upgrade-provider
takes in one required positional argument: the org/repo of the provider, i.e. pulumi/pulumi-docker
.
The flag --upstream-provider-name
is required; it is recommended to set it in the config file while running upgrade-provider
in CI.
Usage:
upgrade-provider <provider> [flags]
Flags:
--allow-missing-docs If true, don't error on missing docs during tfgen.
This is equivalent to setting PULUMI_MISSING_DOCS_ERROR=${! VALUE}.
--experimental Enable experimental features, such as auto token mapping and auto aliasing
-h, --help help for upgrade-provider
--java-version string The version of pulumi-java-gen to target.
--kind strings The kind of upgrade to perform:
- "all": Upgrade the upstream provider and the bridge. Shorthand for "bridge,provider,code,pf".
- "bridge": Upgrade the bridge only.
- "provider": Upgrade the upstream provider only.
- "pf": Upgrade the Plugin Framework only.
- "code": Perform some number of code migrations.
- "check-upstream-version": Determine if we need to upgrade the upstream provider. For use in CI only." (default [all])
--major Upgrade the provider to a new major version.
--migration-opts strings A comma separated list of code migration to perform:
- "autoalias": Apply auto aliasing to the provider.
--pr-assign string A user to assign the upgrade PR to. (default "@me")
--pr-description string Extra text to insert in the generated pull request description.
--pr-reviewers string A comma separated list of reviewers to assign the upgrade PR to.
--remove-plugins Remove all pulumi plugins from cache before running the upgrade.
It is possible that the generated examples may be non-deterministic depending on which
plugins are used if existing versions are present in the cache.
--repo-path string Clone the provider repo to the specified path.
--target-bridge-version ref The desired bridge version to upgrade to. Git hash references permitted. (default <latest>)
--target-pulumi-version ref Upgrade the provider to the passed pulumi/{pkg,sdk} version.
If no version is passed, the pulumi/{pkg,sdk} version will track the bridge
--target-version string Upgrade the provider to the passed version.
If the passed version does not exist, an error is signaled.
--upstream-provider-name string The name of the upstream provider.
Required unless running from provider root and set in upgrade-config.yml.
--upstream-provider-org string The name of the upstream provider's GitHub organization'.
A typical run for a patched provider with an upgrade configuration file will look like this:
❯ upgrade-provider pulumi/pulumi-snowflake
---- Discovering Repository ----
- Ensure 'github.com/pulumi/pulumi-snowflake'
- ✓ Expected Location: /Users/ianwahbe/go/src/github.com/pulumi/pulumi-snowflake
- ✓ Downloading: done
- ✓ Validating: /Users/ianwahbe/go/src/github.com/pulumi/pulumi-snowflake
- pull default branch
- ✓ /usr/local/bin/git ls-remote --heads origin: done
- ✓ finding default branch: master
- ✓ /usr/local/bin/git checkout master: done
- ✓ /usr/local/bin/git pull origin: done
- ✓ Upgrade version: 0.56.3
- ✓ Repo kind: plain
---- Upgrading Provider ----
- Ensure Branch
- ✓ /usr/local/bin/git branch: done
- ✓ Already exists: no
- ✓ /usr/local/bin/git checkout -b upgrade-terraform-provider-snowflake-to-v0.56.3: done
- ✓ /usr/local/bin/git checkout upgrade-terraform-provider-snowflake-to-v0.56.3: done
- ✓ /usr/local/bin/go get -u github.com/pulumi/pulumi-terraform-bridge/v3: done
- ✓ Lookup Tag SHA: 9c69643a31d91d0f3d249f7aea3beeefc53880ae
- ✓ /usr/local/bin/go get github.com/Snowflake-Labs/terraform-provider-snowflake@9c6...: done
- ✓ /usr/local/bin/go mod tidy: done
- ✓ /Users/ianwahbe/go/bin/pulumi plugin rm --all --yes: done
- ✓ /usr/bin/make tfgen: done
- ✓ /usr/local/bin/git add --all: done
- ✓ /usr/local/bin/git commit -m make tfgen: done
- ✓ /usr/bin/make build_sdks: done
- ✓ /usr/local/bin/git add --all: done
- ✓ /usr/local/bin/git commit -m make build_sdks: done
- Open PR
- ✓ /usr/local/bin/git push --set-upstream origin upgrade-terraform-provider-snowfla...: done
- ✓ /usr/local/bin/gh pr create --assignee @me --base master --head upgrade-terrafor...: done
- Self Assign Issues
- ✓ /usr/local/bin/gh issue edit 183 --add-assignee @me: done
- ✓ /usr/local/bin/gh issue edit 182 --add-assignee @me: done
- ✓ /usr/local/bin/gh issue edit 181 --add-assignee @me: done
If the process succeeds, you can go to GitHub and find a pull request opened on your behalf.
The process will fail where manual intervention is required. The failed step will have a
useful error message that should tell you how to address the problem. If a subprocess like
git
or make
fails, its output will be printed.
To fix an error:
Repeat as necessary for a working upgrade.
Ensure you have an upgrade-config.yml
set in the root of your provider:
upstream-provider-name: terraform-provider-snowflake
upstream-provider-org: Snowflake-Labs
Add the Pulumi Upgrade Provider Action to your publishing workflow(s):
- name: Call upgrade provider action
uses: pulumi/pulumi-upgrade-provider-action@v0.0.4
upgrade-provider
defines pipelines, where a pipeline is a set of synchronous and ordered
steps. This leverages the step
module in the repo. The hope is that each pipeline can be
self-documenting. This is the pipeline for running an upgrade after any go.mod
fixes are
already applies:
ok = step.Run(step.Combined("Upgrading Provider",
append(steps,
step.Cmd(exec.CommandContext(ctx, "go", "mod", "tidy")).In(&providerPath),
step.Cmd(exec.CommandContext(ctx, "pulumi", "plugin", "rm", "--all", "--yes")),
step.Cmd(exec.CommandContext(ctx, "make", "tfgen")).In(&path),
step.Cmd(exec.CommandContext(ctx, "git", "add", "--all")).In(&path),
step.Cmd(exec.CommandContext(ctx, "git", "commit", "-m", "make tfgen")).In(&path),
step.Cmd(exec.CommandContext(ctx, "make", "build_sdks")).In(&path),
step.Cmd(exec.CommandContext(ctx, "git", "add", "--all")).In(&path),
step.Cmd(exec.CommandContext(ctx, "git", "commit", "-m", "make build_sdks")).In(&path),
step.Cmd(exec.CommandContext(ctx, "git", "push", "--set-upstream", "origin", branchName)).In(&path),
)...))
upgrade-provider
executes the same process to upgrade a Pulumi provider as a manual
upgrade. The basic pipeline goes as follows:
If the provider references a pulumi owned fork in it's provider/go.mod:
Then the basic provider upgrade is performed:
make tfgen
and check in the result.make build_sdks
and check in the result.If shim
is a subfolder of provider
, then upgrades will be performed in shim
.
A configuration file .upgrade-config.{yml/json}
may be defined within the provider directory.
Values include:
upstream-provider-name
: The name of the upstream provider repo, i.e. terraform-provider-docker
experimental
: Whether to enable experimental pulumi-terraform-bridge
features https://github.com/pulumi/pulumi-terraform-bridge/tree/master/pkg/tfbridge/x. Value must be [true, false (default)].remove-plugins
: Whether to clear all Pulumi plugins from cache before running the upgrade. It is possible that the generated examples may be non-deterministic depending on which plugins are used if existing versions are present in the cache. Values must be [true, false (default)].pr-reviewers
: A comma separated list of reviewers to assign the upgrade PR to.pr-assign
: A user to assign the upgrade PR to (default: @me
).Use PULUMI_REPLAY=logs.json upgrade-provider...
to record logs to use in replay tests like this.
upgrade-provider
should inform the user what it's doing. If
something breaks, the user should be able to diagnose and complete the process on their
own.upgrade-provider
, regardless of where the
user is in an upgrade.upgrade-provider
should make no effort to solve ambiguous
problems, such as build conflicts. If the next step isn't obvious, fail over to the
human operator.