use v2 proxy in `gcloud sql connect`

[aimichal]

Question

When I connect to a Cloud SQL instance with the "gcloud" command, like this:

$ gcloud beta sql connect INSTANCE --user=USER --database=SOME_DB

I see it starting the Cloud SQL Proxy like this:

Starting Cloud SQL Proxy: [/Users/michalg/google-cloud-sdk/bin/cloud_sql_proxy -instances ... -credential_file ...]]

This worked with version 1.33.2, but does not work with version 2.0.0.

The reason is that the argument -instances is now --instances, and using -instances shows the error:

Error: unknown shorthand flag: 'n' in -nstances

(There may be other incompatibilities as well.)

If this is a bug with the "gcloud" command, where should I file the issue? I looked for a while and couldn't find a place, so if you have a pointer, I'll gladly re-post there.

Additional Context

I can't see others bringing this up in the issues; searching for "shorthand" in https://github.com/GoogleCloudPlatform/cloudsql-proxy/issues shows no matching open or closed issues.

In the StackOverflow discussion about this problem, https://stackoverflow.com/questions/75465023/error-unknown-shorthand-flag-n-in-nstances-when-trying-to-connect-google, but there's no mention of the "gcloud" command triggering this problem.

[jackwotherspoon]

Hi @aimichal thanks for the question! 😄

Currently the gcloud sql connect API wraps and uses the invocation of the v1 proxy which is why you are seeing issues with it while attempting to use it with the v2 proxy.

We need to make a few more slight changes to v2 before we can port/sync it over to be used with gcloud. I have updated this issue to a feature request to help track the work and progress.

We hope to have this work done sometime in q2 so that users can leverage the v2 proxy with the gcloud sql connect APIs.

[aimichal]

Thanks for confirming this, @jackwotherspoon! Good to know work is in progress. 👍

In the mean time, we'll tell folks in our org who need to use gcloud sql connect to install v1.33.2 of the proxy.

[wyardley]

Also, would be great if gcloud components could track a more up to date version. I created this issue tracker issue to request it: https://issuetracker.google.com/issues/274849933

[enocom]

You've to the right place -- we're the team that updates that component. We're working on generally upgrading gcloud components to use v2 across the board. We're a small team and there are some other high priorities, but this is coming up the list.

[enocom]

Generally, I'd recommend getting the Proxy from the releases page here, as gcloud will probably always be a minor version or two behind.

[wyardley]

You've to the right place -- we're the team that updates that component. We're working on generally upgrading gcloud components to use v2 across the board.

Great! (Though in this case, was just asking for the latest 1.3.x for now)

Generally, I'd recommend getting the Proxy from the releases page here, as gcloud will probably always be a minor version or two behind.

@enocom yes, but in an environment where the user is unable or unwilling to bypass the OS X restrictions on running unsigned binaries, doing this with unsigned binaries is sometimes challenging.

If the answer is always to "just" download the binary from the GH release (as suggested in https://github.com/GoogleCloudPlatform/cloud-sql-proxy/issues/95), vs. publishing a homebrew recipe for it or keeping the version in the gcloud CLI up to date, then Google may want to invest in signing / notarizing the Apple binaries properly, which I don't think is the case today.

[context: we were suggested to try the newer version in this issue, though in the end, using it didn't "fix" the problem]

[wyardley]

I realize this is a bit of a digression; I opened #1712 to request the signing.

[enocom]

Sounds good. Thanks for the feature request. I'm running on Linux so don't know -- does the gcloud components path cause any headaches for macOS users with the need to notarize binaries?

[wyardley]

I'm running on Linux so don't know -- does the gcloud components path cause any headaches for macOS users with the need to notarize binaries?

Can move this discussion to the other thread if that makes more sense.

I'm using a homebrew / cask install of thegcloud CLI, and so, not sure how / why (since IIRC that's installing it directly as a gcloud component vs. from anything that might be signed by homebrew), but doesn't seem to cause issues, at least for me, when installed that way (maybe homebrew already sets up some kind of broad exemption for the path it's part of?). Other than having to add the resulting binary (that ends up in a weird location) to my $PATH, it works fine.

% xattr -l `which cloud_sql_proxy`
% codesign -dv `which cloud_sql_proxy`
Executable=/opt/homebrew/Caskroom/google-cloud-sdk/latest/google-cloud-sdk/bin/cloud_sql_proxy
Identifier=a.out
Format=Mach-O thin (arm64)
CodeDirectory v=20400 size=129502 flags=0x20002(adhoc,linker-signed) hashes=4044+0 location=embedded
Signature=adhoc
Info.plist=not bound
TeamIdentifier=not set
Sealed Resources=none
Internal requirements=none
% codesign -dv bin/cloud_sql_proxy    
Executable=/Users/xxx/bin/cloud_sql_proxy
Identifier=a.out
Format=Mach-O thin (arm64)
CodeDirectory v=20400 size=135742 flags=0x20002(adhoc,linker-signed) hashes=4239+0 location=embedded
Signature=adhoc
Info.plist=not bound
TeamIdentifier=not set
Sealed Resources=none
Internal requirements=none

There is a process to allow unsigned binaries, if your system gives you permissions to do it. On my work machine, I also had to manually run xattr -d com.apple.quarantine cloud_sql_proxy to actually be able to run it, even after giving it permissions.

As best I understand, the preferred / best practice is to sign the binaries, though I see Signature=adhoc on most of the third party go binaries that I use, so could be wrong.

[enocom]

Thanks for the information. This is super helpful. I'll look into getting a properly signed binary in place. Until then we'll also be working on upgrading to v2 in gcloud.

[enocom]

FYI V2 Proxy is now available as a component in gcloud.

[aimichal]

I just helped someone connect to a Cloud SQL database from a new macOS laptop.

The command gcloud --project xxx-project-name beta sql connect xxx-db-name --user=someuser --database somedb --port 9990 lead us to download and install the latest Cloud SQL Proxy (2.x line) and the same problem occurred (unknown shorthand flag: 'n' in -nstances).

We overwrote the 2.x version of the Cloud SQL Proxy with the 1.x line and the above gcloud command started working.

Is there a schedule when gcloud's sql connect will be updated to work with version 2 of the Cloud SQL Proxy?

[jackwotherspoon]

Is there a schedule when gcloud's sql connect will be updated to work with version 2 of the Cloud SQL Proxy?

@aimichal Yes, I am beginning the work to move gcloud beta sql connect over to the v2 Cloud SQL Proxy in the coming weeks. 👍