pulumi / pulumi-gcp

A Google Cloud Platform (GCP) Pulumi resource package, providing multi-language access to GCP
Apache License 2.0
179 stars 52 forks source link

Docs suggest invalid operation for `gcp.sql.User` #1870

Open banool opened 5 months ago

banool commented 5 months ago

What happened?

I followed the guide as per the docs: https://www.pulumi.com/registry/packages/gcp/api-docs/sql/user/.

    const primaryInstance = (this.primaryInstance = new gcp.alloydb.Instance(
      prefixName("primary"),
      {
        instanceId: prefixName("primary"),
        cluster: this.cluster.id,
        machineConfig: {
          cpuCount: config.cpuCount ?? 4,
        },
        instanceType: "PRIMARY",
        databaseFlags: {
          "google_db_advisor.enabled": "on",
          ...config.dbFlags,
        },
        labels: { ...commonAlloyDBLabels, instance_id: prefixName("primary") },
      },
      {
        parent: this,
        dependsOn: gcpServices.servicenetworking,
      },
    ));

      new gcp.sql.User(prefixName(`read-only`), {
        name: "readonly",
        // Despite what the docs imply, using primaryInstance.name here is invalid.
        instance: primaryInstance.name,
        password: readPoolUserSecrets.initialPassword.result,
        deletionPolicy: "ABANDON",
        type: "BUILT_IN",
      });

When you do this, you get the following error:

error: 1 error occurred:
        * Error, failed to insert user readonly into instance projects/aptos-data-staging/locations/us-central1/clusters/token-management-testnet/instances/token-management-testnet-primary: googleapi: Error 400: Invalid request: Invalid full instance name (aptos-data-staging:projects/aptos-data-staging/locations/us-central1/clusters/token-management-testnet/instances/token-management-testnet-primary).., invalid

This is the log of the resource it is trying to create:

    + gcp:sql/user:User: (create)
        [urn=urn:pulumi:token-management-staging::aptos-indexer-processors::gcp:sql/user:User::token-management-testnet-read-only]
        [provider=urn:pulumi:token-management-staging::aptos-indexer-processors::pulumi:providers:gcp::default_7_7_0::d379f00f-ca7b-4b62-92fc-87e0329e3727]
        deletionPolicy: "ABANDON"
        instance      : "projects/aptos-data-staging/locations/us-central1/clusters/token-management-testnet/instances/token-management-testnet-primary"
        name          : "readonly"
        password      : [secret]
        type          : "BUILT_IN"

It seems like name returns the fully qualified resource ID, not just the instance name. Weirdly enough if I just use the instance name it also doesn't work, I get a 403, which if this answer is right, implies the instance name isn't real, which makes me think the fully qualified resource ID is indeed actually what we want: https://stackoverflow.com/q/45481153/3846032. If so though, something else is amiss.

Example

See above.

Output of pulumi about

CLI          
Version      3.111.1
Go Version   go1.22.1
Go Compiler  gc

Plugins
NAME    VERSION
nodejs  unknown

Host     
OS       darwin
Version  14.4
Arch     arm64

This project is written in nodejs: executable='/Users/dport/Library/Caches/fnm_multishells/16017_1710679214113/bin/node' version='v20.11.1'

Backend        
Name           pulumi.com
URL            https://app.pulumi.com/banool
User           banool
Organizations  banool, aptos-labs
Token type     personal

Pulumi locates its logs in /var/folders/w_/0mh2yzrs5fx2hknmdqxpz40w0000gn/T/ by default
warning: Failed to get information about the Pulumi program's dependencies: could not find either /Users/dport/a/internal-ops/infra/apps/aptos-indexer-processors/yarn.lock or /Users/dport/a/internal-ops/infra/apps/aptos-indexer-processors/package-lock.json
warning: Failed to get information about the current stack: No current stack

Additional context

No response

Contributing

No response

banool commented 5 months ago

I'm using version 7.7.0 of the @pulumi/gcp provider.

aureq commented 5 months ago

@banool I believe there are a few things worth mentioning here.

  1. You probably want to use primaryInstance.instanceId to reference the database.
  2. It's the IAM service that will insert the user into your AlloyDB (see 3). So you'll have to grant permissions somehow (perhaps you need to enable alloydb.iam_authentication) (see 5).
  3. I'm wondering why you're not using alloydb.User instead to create a BUILT_IN user?
  4. On the previous point, it may be helpful to use deletedWith resource option to speed up the instance deletion.
  5. I've tried to use alloydb.iam_authentication, but probably because I don't have the necessary permissions.

Side note: For one user, I used defaultCluster.name and the other primaryInstance.instanceId.

import * as pulumi from "@pulumi/pulumi";
import * as gcp from "@pulumi/gcp";

const defaultNetwork = new gcp.compute.Network("alloydb-network", {
    name: "alloydb-network"
});

const defaultCluster = new gcp.alloydb.Cluster("alloydb-cluster", {
    clusterId: "alloydb-cluster",
    location: "australia-southeast1",
    networkConfig: {
        network: defaultNetwork.id
    },
    initialUser: {
        password: "cluster_secret",
    },
});
const privateIpAlloc = new gcp.compute.GlobalAddress("private_ip_alloc", {
    name: "alloydb-ip-alloc",
    addressType: "INTERNAL",
    purpose: "VPC_PEERING",
    prefixLength: 16,
    network: defaultNetwork.id,
});

const vpcConnection = new gcp.servicenetworking.Connection("vpc_connection", {
    network: defaultNetwork.id,
    service: "servicenetworking.googleapis.com",
    reservedPeeringRanges: [privateIpAlloc.name],
});

const primaryInstance = new gcp.alloydb.Instance("primary", {
    instanceId: "alloydb-instance",
    cluster: defaultCluster.name,
    machineConfig: {
        cpuCount: 4,
    },
    instanceType: "PRIMARY",
    databaseFlags: {
        "alloydb.iam_authentication": "on"
    },
},{
    dependsOn: vpcConnection,
});

const user1 = new gcp.alloydb.User("alloydb-user", {
    cluster: defaultCluster.name,
    userId: "alloydb-user",
    userType: "ALLOYDB_BUILT_IN",
    password: "dwlkemdwlkjedwlkjmoihjoijkmcvewolkmdwokme",
    databaseRoles: ["alloydbiamuser"]
}, {
    deletedWith: primaryInstance
});

// // This code requires additional work and doesn't work.
// // The error returned is "Error, failed to insert user sql-user into instance alloydb-instance: googleapi: Error 403: The client is not authorized to make this request., notAuthorized"
// const user2 = new gcp.sql.User("sql-user", {
//     name: "sql-user",
//     instance: primaryInstance.instanceId,
//     password: "dkjmoihjoijkmcvewolkmd9w8eud98w123wedfdvcdwokme",
//     deletionPolicy: "ABANDON",
//     type: "BUILT_IN",
// }, {
//     deletedWith: primaryInstance
// });
banool commented 5 months ago

Hey, thanks for the quick response!

I was able to get it to work with just 3:

new gcp.alloydb.User(prefixName(`read-only`), {
  userId: READ_POOL_USERNAME,
  password: readPoolUserSecrets.initialPassword.result,
  userType: "ALLOYDB_BUILT_IN",
  databaseRoles: ["pg_read_all_data"],
  cluster: this.cluster.name,
});

Makes total sense to use an alloydb-specific user class, I just didn't know it exists. I suppose I just googled "pulumi sql user" and that's what came up. Perhaps a warning at the top of the docs there that points you to the db-specific user classes would be good, since the docs as they are don't work for AlloyDB (presumably you have to make some of the other changes you mentioned, such as using instanceId rather than name).

Thanks!