open-telemetry / opentelemetry-collector

OpenTelemetry Collector
https://opentelemetry.io
Apache License 2.0
4.46k stars 1.47k forks source link

Extract storage configuration for components into a separate module #5832

Open swiatekm opened 2 years ago

swiatekm commented 2 years ago

Is your feature request related to a problem? Please describe. Components need to use storage extensions for various purposes. There should be a standard way of configuring storage for a component, given the potential for multiple storage extensions coexisting.

See the following contrib issue for examples and some discussion: https://github.com/open-telemetry/opentelemetry-collector-contrib/issues/10915.

Describe the solution you'd like We should add a configstorage module similar to configauth which defines the configuration, and which other components can then use. For the actual configuration, explicitly setting the storage like it's done here: https://github.com/open-telemetry/opentelemetry-collector/pull/5784 seems to be the consensus. For example:

extensions:
  filestorage/example1: {}
  filestorage/example2: {}
receivers:
  filelog:
    storage: filestorage/example1
exporters:
  otlphttp:
    endpoint: <ENDPOINT>
    sending_queue:
      enabled: true
      storage: filestorage/example2

Additional context The following contrib issue was where the discussion started: https://github.com/open-telemetry/opentelemetry-collector-contrib/issues/10915. Additional discussion in the persistent queue PR: https://github.com/open-telemetry/opentelemetry-collector/pull/5784#discussion_r936009614 Impromptu Collector SIG notes on this topic: https://github.com/open-telemetry/opentelemetry-collector-contrib/pull/12871#issuecomment-1204198890

swiatekm commented 5 months ago

Components in core which currently do this:

Components in contrib which currently do this: