grafana / alloy

OpenTelemetry Collector distribution with programmable pipelines
https://grafana.com/oss/alloy
Apache License 2.0
982 stars 103 forks source link

The machine node connection is automatically closed after startup #1160

Open wang-qijia opened 4 days ago

wang-qijia commented 4 days ago

What's wrong?

After starting alloy for 1 second, the console of the machine node is automatically closed.

Steps to reproduce

  1. Start ./alloy-linux-amd64 run cfg.river --server.http.listen-addr=0.0.0.0:8802
  2. The console automatically closes the connection and the machine node needs to be restarted to recover

System information

5.10.134-16.1.al8.x86_64

Software version

v1.2.0

Configuration

discovery.process "all" {
  refresh_interval = "5s"
  discover_config {
    commandline = true
  }
}

discovery.relabel "java" {
    targets = discovery.process.all.targets
    rule {
        action = "replace"
        regex = "app=([^- ]+)"
        replacement = "java-ser"
        source_labels = ["__meta_process_commandline"]
        target_label = "service_name"
    }
}

pyroscope.java "java" {
  profiling_config {
    interval = "15s"
    cpu = true
    lock = "1000ms"
    sample_rate = 10
  }
  forward_to = [pyroscope.write.example.receiver]
  targets = discovery.relabel.java.output
}

pyroscope.write "example" {
  endpoint {
    url = "http://127.0.0.1:4040"
  }
}

Logs

ts=2024-06-27T11:22:42.871038297Z level=info "boringcrypto enabled"=false
ts=2024-06-27T11:22:42.871070007Z level=info msg="running usage stats reporter"
ts=2024-06-27T11:22:42.871080696Z level=info msg="starting complete graph evaluation" controller_path=/ controller_id="" trace_id=fcb813d4c0d70bafed8b879308ffca34
ts=2024-06-27T11:22:42.871097679Z level=info msg="finished node evaluation" controller_path=/ controller_id="" trace_id=fcb813d4c0d70bafed8b879308ffca34 node_id=tracing duration=4.559µs
ts=2024-06-27T11:22:42.871106656Z level=info msg="finished node evaluation" controller_path=/ controller_id="" trace_id=fcb813d4c0d70bafed8b879308ffca34 node_id=otel duration=992ns
ts=2024-06-27T11:22:42.871113799Z level=info msg="finished node evaluation" controller_path=/ controller_id="" trace_id=fcb813d4c0d70bafed8b879308ffca34 node_id=pyroscope.write.example duration=152.897µs
ts=2024-06-27T11:22:42.871121012Z level=info msg="finished node evaluation" controller_path=/ controller_id="" trace_id=fcb813d4c0d70bafed8b879308ffca34 node_id=discovery.process.all duration=18.665µs
ts=2024-06-27T11:22:42.871127945Z level=info msg="finished node evaluation" controller_path=/ controller_id="" trace_id=fcb813d4c0d70bafed8b879308ffca34 node_id=discovery.relabel.java duration=72.446µs
ts=2024-06-27T11:22:42.871134568Z level=info msg="finished node evaluation" controller_path=/ controller_id="" trace_id=fcb813d4c0d70bafed8b879308ffca34 node_id=pyroscope.java.java duration=71.638683ms
ts=2024-06-27T11:22:42.871141421Z level=info msg="finished node evaluation" controller_path=/ controller_id="" trace_id=fcb813d4c0d70bafed8b879308ffca34 node_id=remotecfg duration=55.695µs
ts=2024-06-27T11:22:42.871148083Z level=info msg="finished node evaluation" controller_path=/ controller_id="" trace_id=fcb813d4c0d70bafed8b879308ffca34 node_id=livedebugging duration=8.826µs
ts=2024-06-27T11:22:42.871169514Z level=info msg="finished node evaluation" controller_path=/ controller_id="" trace_id=fcb813d4c0d70bafed8b879308ffca34 node_id=labelstore duration=5.791µs
ts=2024-06-27T11:22:42.871177138Z level=info msg="applying non-TLS config to HTTP server" service=http
ts=2024-06-27T11:22:42.871186386Z level=info msg="finished node evaluation" controller_path=/ controller_id="" trace_id=fcb813d4c0d70bafed8b879308ffca34 node_id=http duration=7.444µs
ts=2024-06-27T11:22:42.871194831Z level=info msg="finished node evaluation" controller_path=/ controller_id="" trace_id=fcb813d4c0d70bafed8b879308ffca34 node_id=ui duration=2.895µs
ts=2024-06-27T11:22:42.871207726Z level=info msg="finished node evaluation" controller_path=/ controller_id="" trace_id=fcb813d4c0d70bafed8b879308ffca34 node_id=cluster duration=2.915µs
ts=2024-06-27T11:22:42.871217624Z level=info msg="finished node evaluation" controller_path=/ controller_id="" trace_id=fcb813d4c0d70bafed8b879308ffca34 node_id=logging duration=185.339µs
ts=2024-06-27T11:22:42.871227042Z level=info msg="finished complete graph evaluation" controller_path=/ controller_id="" trace_id=fcb813d4c0d70bafed8b879308ffca34 duration=72.288466ms
ts=2024-06-27T11:22:42.871345705Z level=info msg="scheduling loaded components and services"
ts=2024-06-27T11:22:42.871426648Z level=info msg="starting cluster node" peers="" advertise_addr=127.0.0.1:8802
ts=2024-06-27T11:22:42.871646471Z level=info msg="peers changed" new_peers=ALIBJ2-IM-Test-DiaoChan-Service-8C16G01-10-41-1-1
ts=2024-06-27T11:22:42.871879199Z level=info msg="now listening for http traffic" service=http addr=0.0.0.0:8802
ts=2024-06-27T11:22:42.907043074Z level=error component_path=/ component_id=pyroscope.java.java pid=713 err="failed to copy libasyncProfiler.so: failed to open proc root /proc/713/root: open /proc/713/root: no such file or directory"
ts=2024-06-27T11:22:42.921796807Z level=error component_path=/ component_id=pyroscope.java.java pid=1256 err="failed to copy libasyncProfiler.so: failed to open proc root /proc/1256/root: open /proc/1256/root: no such file or directory"

Log of failed machine connections: 
error: gateway dial addr failed: ssh: rejected: connect failed (Connection refused)
Receive Connection closed
Connection websocket closed