clustering: allow cluster-aware component scheduling

grafana / alloy

OpenTelemetry Collector distribution with programmable pipelines

https://grafana.com/oss/alloy

Apache License 2.0

1.37k stars 199 forks source link

clustering: allow cluster-aware component scheduling #399

Open tpaschalis opened 1 year ago

tpaschalis commented 1 year ago

Request

We've discussed about introducing a second use case for clustering; fine-grained component scheduling.

This would allow components to opt-in to a new mechanism which would enable running N replicas of a component inside of a cluster, rebalancing whenever the cluster state changes.

Use case

Running exporters only on a subset of peers
Running service discovery centrally and distributing the results within other peers

github-actions[bot] commented 1 year ago

This issue has been automatically marked as stale because it has not had any activity in the past 30 days. The issue will not be closed automatically, but a label will be added to it for tracking purposes.

If the opened issue is a bug, check if newer releases have fixed the issue. If the issue is no longer relevant, please feel free to close it. Thank you for your contributions!

hervenicol commented 1 month ago

Maybe it could be solved with conditionals? Here is an issue stating that it's currently missing but could be useful 😅 https://github.com/grafana/alloy/issues/1079