search

googleapis / google-cloud-go

Google Cloud Client Libraries for Go.
https://cloud.google.com/go/docs/reference
Apache License 2.0
3.72k stars 1.28k forks source link

metadata: support signaling "not on GCE" #4920

Open stevo-f3 opened 3 years ago

stevo-f3 commented 3 years ago

Is your feature request related to a problem? Please describe. Multiple components that we have running exactly same on multi-cloud/onprem environments have transitive dependency on this library, and metadata lookups for "on GCP" test keeps spamming our DNS servers.

Describe the solution you'd like Please support, not only explicitly signaling via GCE_METADATA_HOST env var that the component is running on GCP (see https://github.com/googleapis/google-cloud-go/blob/ea00264428137471805f2ec67f04f3a5a42928fa/compute/metadata/metadata.go#L117-L121), but also signaling, via another env var or special value of existing one, that environment is not GCE one.

Describe alternatives you've considered I've considered configuring local DNS resolvers to treat *.internal as local domain, not to be resolved by DNS servers. Problem is we have some .internal domains which have to be resolved by DNS servers. Configuring more specific domain (like metadata.google.internal) as local or as ignored/refused is not supported on local resolver that we have. Replacing local resolver to a more capable one is considered to be way much more involving compared to updating dependency to this go library and setting an environment variable.

Additional context N/A

codyoss commented 3 years ago

Hey @stevo-f3 thanks for the feature request. I see how this could be useful in non-GCE environments.

Out of curiosity how are you authenticating with your services? The metadata check is the last method checked in our "Application Default Credential" flow: FindDefaultCredentials. I would think if other valid credential flows were found you would not hit this check.

stevo-f3 commented 3 years ago

Will have to check on the auth. Thinking out loud, can there be valid credential flow on non-GCE environment?

codyoss commented 3 years ago

@stevo-f3 Yes, a couple come to mind:

  1. A Service account key file can be used to authenticate.
  2. User credential files (gcloud auth login)
  3. Also external accounts using identity federation with STS.
codyoss commented 5 months ago

I have re-rasied this issue to internal teams.

edaniels commented 3 weeks ago

Any word on this @codyoss? We've worked around it with some special casing in our codebase but it'd be nice to remove. Thank you!

codyoss commented 3 weeks ago

I do not have a timeline, but we do intend to do something about this issue in the future.

edaniels commented 3 weeks ago

I do not have a timeline, but we do intend to do something about this issue in the future.

Perfect. Thanks Cody