We are considering to enforce using pulumi convert for the build-time Terraform example translation. Requesting feedback from bridged provider maintainers as this change will impact bridged provider builds and CI.
Background
As part of running make tfgen in a bridged provider, the build will attempt to locate Terraform examples, correlate them with the relevant Resource or Data Source documentation, and translate them to Pulumi examples. The result of this process is published in the Pulumi Package Schema informing the documentation rendering in the Pulumi registry, and also informs generated code for the SDKs in each Pulumi- supported language (Node, Python, Go, .NET and Java).
Currently the default method of translating the examples relied on linking in deprecated translation code into the build process directly. The improved method uses pulumi convert which invokes pulumi-converter-terraform under the hood. This change is looking at deprecating the default method as it is falling behind in functionality compared to pulumi convert and is not receiving fixes.
ensure Terraform converter plugin is installed (required)
ensure Terraform converter plugin and any plugins referenced from examples are pinned (recommended)
Specifically pins are recommended for providers that build frequently and commit generated SDKs such as sdk/python to source control. If plugin references are left floating, out of band changes such as new releases of Pulumi providers may render the generated SDK code stale with respect to re-running make tfgen, which may negatively affect build hermeticity. These concerns may be ignored for providers that build infrequently or do not strictly enforce hermetic SDK builds.
Testing provider builds
Builds utilizing pulumi convert for example rendering can be currently tested by setting the PULUMI_CONVERT=1 environment variable. In the future bridged providers will opt into this behavior by default instead of gating it by an environment variable.
Example provider builds
Pulumi has finished migrating all internally managed bridged providers to use pulumi convert.
An example in pulumi-aws provider demonstrates pinning:
It also opts into pulumi convert example rendering with the PULUMI_CONVERT=1 environment variable. In the future bridged providers will opt into this behavior by default instead of gating it by an environment variable.
Impact
Based on the experience of rolling the change out internally, the change is beneficial to the quality of the example conversion process:
We are considering to enforce using
pulumi convertfor the build-time Terraform example translation. Requesting feedback from bridged provider maintainers as this change will impact bridged provider builds and CI.Background
As part of running
make tfgenin a bridged provider, the build will attempt to locate Terraform examples, correlate them with the relevant Resource or Data Source documentation, and translate them to Pulumi examples. The result of this process is published in the Pulumi Package Schema informing the documentation rendering in the Pulumi registry, and also informs generated code for the SDKs in each Pulumi- supported language (Node, Python, Go, .NET and Java).Currently the default method of translating the examples relied on linking in deprecated translation code into the build process directly. The improved method uses
pulumi convertwhich invokes pulumi-converter-terraform under the hood. This change is looking at deprecating the default method as it is falling behind in functionality compared topulumi convertand is not receiving fixes.All bridged providers managed by Pulumi have finished migrating to the new method.
Preparing the provider build environment
A provider maintainer needs to:
pulumiCLI is installed (required)Specifically pins are recommended for providers that build frequently and commit generated SDKs such as sdk/python to source control. If plugin references are left floating, out of band changes such as new releases of Pulumi providers may render the generated SDK code stale with respect to re-running
make tfgen, which may negatively affect build hermeticity. These concerns may be ignored for providers that build infrequently or do not strictly enforce hermetic SDK builds.Testing provider builds
Builds utilizing
pulumi convertfor example rendering can be currently tested by setting the PULUMI_CONVERT=1 environment variable. In the future bridged providers will opt into this behavior by default instead of gating it by an environment variable.Example provider builds
Pulumi has finished migrating all internally managed bridged providers to use
pulumi convert.An example in pulumi-aws provider demonstrates pinning:
https://github.com/pulumi/pulumi-aws/blob/master/Makefile#L114
The Makefile pins Pulumi CLI version, the Terraform converter version and several related plugins as in:
It also opts into
pulumi convertexample rendering with the PULUMI_CONVERT=1 environment variable. In the future bridged providers will opt into this behavior by default instead of gating it by an environment variable.Impact
Based on the experience of rolling the change out internally, the change is beneficial to the quality of the example conversion process:
There are a few known minor regressions:
Issues can be filed against https://github.com/pulumi/pulumi-converter-terraform for further improvements in this area.