search

GoogleCloudPlatform / cloud-sql-proxy

A utility for connecting securely to your Cloud SQL instances
Apache License 2.0
1.25k stars 345 forks source link

Automatic instance discovery isn't supported in v2 Proxy #2243

Closed rstreefland closed 4 weeks ago

rstreefland commented 1 month ago

Bug Description

The automatic instance discovery does not work with v2 of the proxy. It works as expected with v1 of the proxy but I've needed to upgrade to v2 to support automatic IAM database authentication.

Example code (or command)

./cloud-sql-proxy --unix-socket ./cloudsql

Stacktrace

➜  ~ curl -o cloud-sql-proxy https://storage.googleapis.com/cloud-sql-connectors/cloud-sql-proxy/v2.11.3/cloud-sql-proxy.darwin.arm64

  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100 31.3M  100 31.3M    0     0  4247k      0  0:00:07  0:00:07 --:--:-- 4675k
➜  ~ chmod +x cloud-sql-proxy
➜  ~ gcloud auth application-default login
Your browser has been opened to visit:

    REDACTED

Credentials saved to file: [/Users/rhys/.config/gcloud/application_default_credentials.json]

These credentials will be used by any library that requests Application Default Credentials (ADC).

Quota project "REDACTED" was added to ADC which can be used by Google client libraries for billing and quota. Note that some services may still bill the project owning the resource.
➜  ~ ./cloud-sql-proxy --unix-socket ./cloudsql
Error: missing instance_connection_name (e.g., project:region:instance)
Usage:
  cloud-sql-proxy INSTANCE_CONNECTION_NAME... [flags]
  cloud-sql-proxy [command]

Available Commands:
  completion  Generate the autocompletion script for the specified shell
  help        Help about any command
  wait        Wait for another Proxy process to start

Flags:
...

Steps to reproduce?

  1. Install and run v2 of the proxy as per instructions https://cloud.google.com/sql/docs/postgres/connect-auth-proxy#invocations ...

Environment

  1. OS type and version: MacOS 14.1.1
  2. Cloud SQL Proxy version (./cloud-sql-proxy --version): cloud-sql-proxy version 2.11.3+darwin.arm64
  3. Proxy invocation command (for example, ./cloud-sql-proxy --port 5432 INSTANCE_CONNECTION_NAME): ./cloud-sql-proxy --unix-socket ./cloudsql

Additional Details

No response

jackwotherspoon commented 1 month ago

Hi @rstreefland thanks for raising an issue on the Cloud SQL Proxy 😄

You are correct that this is indeed not a supported feature in the Cloud SQL Proxy v2.

@enocom was there a reason this was not ported over to v2?

https://github.com/GoogleCloudPlatform/cloud-sql-proxy/blob/38636926f3aef10331deb91907155d34931d191d/cmd/cloud_sql_proxy/cloud_sql_proxy.go#L259-L263

My assumption would be is that we don't want v2 to be dependent on gcloud internals.

enocom commented 1 month ago

For reference the code is here where v1 Proxy would connect to all Cloud SQL instances within the active gcloud project. We didn't port this feature because it had surprising behavior and prefer explicit configuration.

So, there are two options for v2:

  1. Explicitly list the instances you want to connect to (either as arguments or environment variables)
  2. Use the --fuse flag such that connections are established lazily.

If neither of those fits what you're trying to do @rstreefland, I'd be curious to hear more about your use case.

rstreefland commented 1 month ago

For reference the code is here where v1 Proxy would connect to all Cloud SQL instances within the active gcloud project. We didn't port this feature because it had surprising behavior and prefer explicit configuration.

So, there are two options for v2:

  1. Explicitly list the instances you want to connect to (either as arguments or environment variables)
  2. Use the --fuse flag such that connections are established lazily.

If neither of those fits what you're trying to do @rstreefland, I'd be curious to hear more about your use case.

@enocom Thanks for the explanation! Honestly it's just the lowest effort option when connecting to multiple databases in the same project for local development, but I'm not overly bothered that this doesn't exist in v2 as I could easily replicate it with bash.

What confused me is that the documentation has been updated for v2, but it still suggests that automatic instance discovery is possible: https://cloud.google.com/sql/docs/postgres/connect-auth-proxy#invocations https://cloud.google.com/sql/docs/postgres/sql-proxy#instances-options

jackwotherspoon commented 1 month ago

What confused me is that the documentation has been updated for v2, but it still suggests that automatic instance discovery is possible: https://cloud.google.com/sql/docs/postgres/connect-auth-proxy#invocations https://cloud.google.com/sql/docs/postgres/sql-proxy#instances-options

@rstreefland Great catch! I'll get those removed right away, whoops 🙈

jackwotherspoon commented 4 weeks ago

I have removed the automatic instance discovery samples from the public docs.

Thanks again for catching this @rstreefland 👏

Much appreciated 😄