Simplifying stateful behavior

[VihasMakwana]

Component(s)

No response

Is your feature request related to a problem? Please describe.

Currently, the collector operates in a stateless mode by default, with stateful components storing offsets in memory. However, stateful components should persist their state during shutdown if a storage extension is available.

At present, enabling stateful behavior involves a somewhat lengthy process:

1. Adding a filestorage entry to the extensions stanza in the configuration.
2. Including filestorage under service::extensions.
3. Ensuring that the storage directory exists.
4. Adding storageID: filelogreceiver/xyz to individual components.

It would be beneficial to simplify this process by introducing a feature gate or a single configuration option. With this approach, users could enable stateful mode with a single setting, and the necessary steps would be handled automatically, achieving the same effect as the manual steps described.

Describe the solution you'd like

Here’s one way to tackle the issue:

• Introduce a new configuration provider that provides us with a default filestorage extension.
  • We might also consider including some commonly used extensions, such as healtcheck, pprof, memorylimitter, etc.
• Inject this new configuration.
• Enable stateful receivers to automatically utilize the storage extension when the feature flag is turned on.

This is a high-level overview, and I expect it will require multiple pull requests to implement.

Describe alternatives you've considered

Please share if you have any thoughts!

Additional context

My solution would require https://github.com/open-telemetry/opentelemetry-collector/pull/11046 to be merged or a similar workaround to be implemented to combine lists in config.

[VihasMakwana]

^^ @open-telemetry/collector-contrib-approvers @open-telemetry/collector-contrib-maintainers

[VihasMakwana]

Closing this in favour of https://github.com/open-telemetry/opentelemetry-collector-contrib/issues/35013