Reduce the resource usage of go when compiling `compute`

[iwahbe]

Hello!

• Vote on this issue by adding a 👍 reaction
• If you want to implement this feature, comment to let us know (we'll work with you on design, scheduling, etc.)

Issue details

This is a continuation from https://github.com/pulumi/pulumi-gcp/issues/1283.

Closing #1283 reduced the memory usage by half, but compiling an IaC program should use less than 4GB. We should continue to explore strategies to reduce resource usage while compiling compute.

Suggested strategies

• Break up the compute module:
  • Upside: This keeps with the current upstream -> schema -> code model.
  • Upside: We can apply this easily to new resources.
  • Downside: To actually reduce the size of the compute module will be a major breaking change.
• Move rarely used types to a secondary module:
  • Upside: This is a very minor breaking change in exchange for reducing the size of compute right now.
  • Downside: We need to change the upstream -> schema -> code model for this module.

Affected area/feature

[t0yv0]

We can also remove some supplementary types like being less aggressive with the monomorphic forms (PtrOutput etc).

This bloat is generally not confined to GCP. I wrote a quick script to detect bloat across all our providers.

#!/usr/bin/env bash

set -euo pipefail

providers=$(curl https://raw.githubusercontent.com/pulumi/ci-mgmt/master/provider-ci/providers.json | jq -r .[])

echo "| P | rel | br | pf | pkg | sdk |"
for p in $providers
do
    D="/tmp/pulumi-$p"
    find "$D/sdk/go" -name pulumiTypes1.go
done

find "/Users/t0yv0/code/pulumi-azure-native-sdk" -name pulumiTypes1.go
find "/Users/t0yv0/code/pulumi-aws-native" -name pulumiTypes1.go
find "/Users/t0yv0/code/pulumi-kubernetes" -name pulumiTypes1.go

/tmp/pulumi-akamai/sdk/go/akamai/pulumiTypes1.go
/tmp/pulumi-aws/sdk/go/aws/wafv2/pulumiTypes1.go
/tmp/pulumi-azure/sdk/go/azure/appservice/pulumiTypes1.go
/tmp/pulumi-datadog/sdk/go/datadog/pulumiTypes1.go
/tmp/pulumi-gcp/sdk/go/gcp/compute/pulumiTypes1.go
/tmp/pulumi-oci/sdk/go/oci/database/pulumiTypes1.go
/tmp/pulumi-oci/sdk/go/oci/core/pulumiTypes1.go
/tmp/pulumi-oci/sdk/go/oci/identity/pulumiTypes1.go
/tmp/pulumi-rancher2/sdk/go/rancher2/pulumiTypes1.go
/tmp/pulumi-sumologic/sdk/go/sumologic/pulumiTypes1.go
/Users/t0yv0/code/pulumi-azure-native-sdk/datafactory/v20180601/pulumiTypes1.go
/Users/t0yv0/code/pulumi-azure-native-sdk/datafactory/pulumiTypes1.go
/Users/t0yv0/code/pulumi-azure-native-sdk/network/v20230501/pulumiTypes1.go
/Users/t0yv0/code/pulumi-azure-native-sdk/network/v20230201/pulumiTypes1.go
/Users/t0yv0/code/pulumi-azure-native-sdk/network/v20230601/pulumiTypes1.go
/Users/t0yv0/code/pulumi-azure-native-sdk/network/pulumiTypes1.go
/Users/t0yv0/code/pulumi-azure-native-sdk/network/v20230401/pulumiTypes1.go
/Users/t0yv0/code/pulumi-azure-native-sdk/machinelearningservices/v20230801preview/pulumiTypes1.go
/Users/t0yv0/code/pulumi-azure-native-sdk/machinelearningservices/v20230401preview/pulumiTypes1.go
/Users/t0yv0/code/pulumi-azure-native-sdk/machinelearningservices/pulumiTypes1.go
/Users/t0yv0/code/pulumi-azure-native-sdk/machinelearningservices/v20231001/pulumiTypes1.go
/Users/t0yv0/code/pulumi-azure-native-sdk/machinelearningservices/v20230601preview/pulumiTypes1.go
/Users/t0yv0/code/pulumi-aws-native/sdk/go/aws/quicksight/pulumiTypes1.go