New component: Vault Config Source

[atoulme]

The purpose and use-cases of the new component

This component currently exists in the Splunk OpenTelemetry Collector distribution: https://github.com/signalfx/splunk-otel-collector/tree/main/internal/configsource/vaultconfigsource

Use the Vault config source to retrieve data from Vault and inject it into your collector configuration. It supports:

• Dynamic secrets;
• Key/Value V1 lease hints;
• Key/Value V2 metadata polling;

Example configuration for the component

Under the config_sources: use vault: or vault/<name>: to create a Vault config source. The following parameters are available to customize Vault config sources:

config_sources:
  vault:
    # endpoint is the Vault server address. It is equivalent to the Vault tool
    # environment variable VAULT_ADDR.
    endpoint: http://localhost:8200
    # path is the Vault path to the secret location.
    path: secret/data/kv
    # poll_interval is used only for non-dynamic V2 K/V secret stores. It is
    # the interval in which the config source will check for changes on the
    # data on the given Vault path. Defaults to 1 minute if not specified.
    poll_interval: 90s
    # auth is a section used to indicate the authentication method to be used.
    # Exactly one method must be specified, it must be one of the following:
    # "token", "iam", or "gcp".
    auth:
      # token is used to access the Vault server. It is equivalent to the Vault tool
      # environment variable VAULT_TOKEN.
      token: some_toke_value
      # iam is used on AWS deployments to generate the required Vault token.
      # For details about each of the settings below, see
      # https://github.com/hashicorp/vault/blob/v1.1.0/builtin/credential/aws/cli.go#L148
      iam:
        aws_access_key_id: key_id
        aws_secret_access_key: access_key
        aws_security_token: security_token
        header_value: header_value
        mount: aws
        role: role
      # gcp is used on GCP deployments to generate the required Vault token.
      # For details about each of the settings below, see
      # https://github.com/hashicorp/vault-plugin-auth-gcp/blob/e1f6784b379d277038ca0661606aa8d23791e392/plugin/cli.go#L138
      gcp:
        role: role
        mount: gcp
        credentials: json_string # This setting is not recommended.
        jwt_ext: 10
        service_account: some_account
        project: project_id

If multiple paths are needed create different instances of the config source, example:

config_sources:
    # Assuming that the environment variables VAULT_ADDR and VAULT_TOKEN are the defined
    # and the different secrets are on the same server but at different paths.
    vault/kv:
      endpoint: $VAULT_ADDR
      path: secret/data/kv
      auth:
        token: $VAULT_TOKEN
    vault/db:
      endpoint: $VAULT_ADDR
      path: database/creds/collector_role
      auth:
        token: $VAULT_TOKEN

# Both Vault config sources can be used via their full name. Hypothetical example:
components:
  component_using_vault_kv:
    # Example showing K/V V2, see note below about the '.' usage.
    username: $vault/kv:data.user
    password: $vault/kv:data.password

  component_using_vault_db:
    username: $vault/db:username
    password: $vault/db:password

Note: When using the Key/Value V2 secret engine, all data will be nested under a separate data map within the secret, e.g. data and metadata, to access specific keys specify the "map" and the "key" using a . as separator, eg: data.username.

Telemetry data types supported

N/A

Is this a vendor-specific component?

• [ ] This is a vendor-specific component
• [ ] If this is a vendor-specific component, I am proposing to contribute this as a representative of the vendor.

Sponsor (optional)

No response

Additional context

No response

[github-actions[bot]]

This issue has been inactive for 60 days. It will be closed in 60 days if there is no activity. To ping code owners by adding a component label, see Adding Labels via Comments, or if you are unsure of which component this issue relates to, please ping @open-telemetry/collector-contrib-triagers. If this issue is still relevant, please ping the code owners or leave a comment explaining why it is still relevant. Otherwise, please close it.

[github-actions[bot]]

This issue has been closed as inactive because it has been stale for 120 days with no activity.

[kago-dk]

It would be great to get added.

[rmfitzpatrick]

cc: @MovieStoreGuy based on your sig comment about secrets management.

The current requirement to shim all confmap.Provider settings via resolver uris is tricky for nontrivial configuration though, and reconciling that w/ the proposed config source interface is the main challenge w/ rfes like this tmk.

[MovieStoreGuy]

I would be interested in something like this but perhaps less "custom" is the best way to phrase it.

For example, if someone wanted to adopt this with the collector builder, they would need to do some custom patching in order for it to work.

[arpitjindal97]

+1 for the feature

[atoulme]

@MovieStoreGuy looks like that's orthogonal to the config source provider, you're saying that the builder doesn't support setting additional config source providers. I believe this is tracked here: https://github.com/open-telemetry/opentelemetry-collector/issues/4759

Would you be interested to sponsor this component?

[github-actions[bot]]

This issue has been inactive for 60 days. It will be closed in 60 days if there is no activity. To ping code owners by adding a component label, see Adding Labels via Comments, or if you are unsure of which component this issue relates to, please ping @open-telemetry/collector-contrib-triagers. If this issue is still relevant, please ping the code owners or leave a comment explaining why it is still relevant. Otherwise, please close it.