search

grafana / alloy

OpenTelemetry Collector distribution with programmable pipelines
https://grafana.com/oss/alloy
Apache License 2.0
1.43k stars 210 forks source link

Alloy clustering on Istio doesn’t seem to work #1846

Open yann-soubeyrand opened 1 month ago

yann-soubeyrand commented 1 month ago

What's wrong?

Hi,

I’ve used the facility offered by https://github.com/grafana/alloy/pull/820 to change the name of the port on the cluster service to tcp so that clustering works when deploying Alloy on Istio. However, I can’t get it to work. I’ve had to also change the name of the “normal” service to tcp to get clustering to work.

Steps to reproduce

Deploy Alloy using the Helm chart and the following values:

alloy:
  clustering:
    enabled: true
    portName: tcp # https://istio.io/latest/docs/ops/common-problems/network-issues/#503-error-while-accessing-headless-services

Look at the state of Alloy clustering (from the web UI) and see that the cluster is only composed of the local node.

One can see the following in Istio access logs:

alloy-fcr7h istio-proxy [2024-10-08T11:36:03.412Z] "POST /api/v1/ckit/transport/stream HTTP/2" 0 DR http2.remote_reset - "-" 0 0 10000 - "-" "Go-http-client/2.0" "fff0c295-d5f6-4f6b-b2e1-dc43f694e021" "172.31.91.255:12345" "172.31.91.255:12345" inbound|12345|| 127.0.0.6:49319 172.31.91.255:12345 172.31.66.90:51870 invalid:outbound_.12345_._.alloy-cluster.alloy.svc.cluster.local default

System information

Kubernetes on Linux AMD64 nodes

Software version

Grafana Alloy v1.3.0 deployed using Helm chart v0.6.0

Configuration

No response

Logs

No response

github-actions[bot] commented 1 week ago

This issue has not had any activity in the past 30 days, so the needs-attention label has been added to it. If the opened issue is a bug, check to see if a newer release fixed your issue. If it is no longer relevant, please feel free to close this issue. The needs-attention label signals to maintainers that something has fallen through the cracks. No action is needed by you; your issue will be kept open and you do not have to respond to this comment. The label will be removed the next time this job runs if there is new activity. Thank you for your contributions!