cedwardstesla
commented
2 months ago Would like to second this. Similar to Influxdata Telegraf it would be great if we could specify a directory instead and have otel-col recurse through it and combine the various config files to load.
See Telegraf docs on how they handle it: https://github.com/influxdata/telegraf/blob/master/docs/CONFIGURATION.md#configuration-loading
Component(s)
collector
Is your feature request related to a problem? Please describe.
We have dozens of applications that get deployed to a Windows environment, where each application would have its own configuration for log and metric collection. Some of these applications are also optional, so we cannot assume the list of applications is fixed. Furthermore, we plan to add more applications over time, which means the list of configs will also change. Given this context, we are looking for the ability to install and configure the Open Telemetry Collector once and have it dynamically discover the configuration for each service that is deployed without needing to change the startup arguments of the service as new deploymens occur.
Describe the solution you'd like
Something similar to file_sd_config from the Prometheus world would work well for us.
A simpler solution, which would work very well for us too, is to have the service check if the config path is a directory or a file. If it is a directory, it would grab all files recursively and merge all the configuration as it does now when someone uses multiple config files manually.
Describe alternatives you've considered
The closest way to dynamically load new configs for new deployments is to use a PowerShell script to dynamically loop through folders to find files and update the binary path (startup arguments) of the Open Telemetry Collector service (as per https://github.com/open-telemetry/opentelemetry-collector/pull/4727). This means that the service has to be stopped and restarted with the new args. Also, troubleshooting missing logs/metrics is harder because we need to examine a long list of startup arguments, which are only up-to-date if someone "remembers" to run the script.
Additional context
No response