Closed sokoow closed 11 months ago
edeNFed
commented
11 months ago Hi @sokoow can you please show the logs of the odiglet pod? This is the pod responsible for instrumenting applications and producing the data. This will help us to understand the root cause.
kubectl logs -n odigos-system <odiglet-pod-name>
sokoow
commented
11 months ago ah, I can see it now:
{"level":"info","ts":1698928991.8865356,"caller":"kube/shared.go:110","msg":"updated runtime info","result":"created","name":"gitea","kind":"Deployment","namespace":"default"}
{"level":"info","ts":1698929196.8323047,"caller":"kube/pods.go:48","msg":"pod is not running, removing instrumentation","controller":"pod","controllerGroup":"","controllerKind":"Pod","Pod":{"name":"gitea-6f6dbbb474-hf2kt","namespace":"default"},"namespace":"default","name":"gitea-6f6dbbb474-hf2kt","reconcileID":"5c06ab98-023b-4091-a32a-853bee68b279"}
{"level":"info","ts":1698929197.6842651,"caller":"kube/pods.go:48","msg":"pod is not running, removing instrumentation","controller":"pod","controllerGroup":"","controllerKind":"Pod","Pod":{"name":"gitea-6f6dbbb474-hf2kt","namespace":"default"},"namespace":"default","name":"gitea-6f6dbbb474-hf2kt","reconcileID":"12916ff3-a6ee-4819-833b-3e929b505534"}
{"level":"info","ts":1698929240.7900429,"caller":"ebpf/director.go:47","msg":"Instrumenting process","pid":54903}
{"level":"info","ts":1698929240.7901502,"caller":"ebpf/director.go:47","msg":"Instrumenting process","pid":54918}
{"level":"info","ts":1698929240.7903354,"caller":"opentelemetry/controller.go:101","msg":"Establishing connection to OTLP receiver ..."}
{"level":"info","ts":1698929240.7906945,"caller":"opentelemetry/controller.go:101","msg":"Establishing connection to OTLP receiver ..."}
{"level":"info","ts":1698929240.791354,"caller":"process/analyze.go:109","msg":"No symbols found in binary, trying to find functions using .gosymtab"}
{"level":"error","ts":1698929240.827417,"caller":"ebpf/director.go:58","msg":"instrumentation setup failed","error":"not a Go executable","stacktrace":"github.com/keyval-dev/odigos/odiglet/pkg/ebpf.(*InstrumentationDirector).Instrument.func1\n\t/go/src/github.com/keyval-dev/odigos/odiglet/pkg/ebpf/director.go:58"}
{"level":"error","ts":1698929241.0389721,"caller":"process/analyze.go:93","msg":"Failed to find functions","error":".gosymtab section not found in target binary, make sure this is a Go application","stacktrace":"go.opentelemetry.io/auto/internal/pkg/process.(*Analyzer).Analyze\n\t/root/go/pkg/mod/go.opentelemetry.io/auto@v0.7.0-alpha/internal/pkg/process/analyze.go:93\ngo.opentelemetry.io/auto.NewInstrumentation\n\t/root/go/pkg/mod/go.opentelemetry.io/auto@v0.7.0-alpha/instrumentation.go:88\ngithub.com/keyval-dev/odigos/odiglet/pkg/ebpf.(*InstrumentationDirector).Instrument.func1\n\t/go/src/github.com/keyval-dev/odigos/odiglet/pkg/ebpf/director.go:56"}
{"level":"error","ts":1698929241.0398512,"caller":"ebpf/director.go:58","msg":"instrumentation setup failed","error":".gosymtab section not found in target binary, make sure this is a Go application","stacktrace":"github.com/keyval-dev/odigos/odiglet/pkg/ebpf.(*InstrumentationDirector).Instrument.func1\n\t/go/src/github.com/keyval-dev/odigos/odiglet/pkg/ebpf/director.go:58"}
{"level":"info","ts":1698929256.875302,"caller":"ebpf/director.go:47","msg":"Instrumenting process","pid":54903}
{"level":"info","ts":1698929256.875548,"caller":"ebpf/director.go:47","msg":"Instrumenting process","pid":54918}
{"level":"info","ts":1698929256.877129,"caller":"opentelemetry/controller.go:101","msg":"Establishing connection to OTLP receiver ..."}
{"level":"info","ts":1698929256.8791273,"caller":"opentelemetry/controller.go:101","msg":"Establishing connection to OTLP receiver ..."}
{"level":"error","ts":1698929256.8861096,"caller":"ebpf/director.go:58","msg":"instrumentation setup failed","error":"not a Go executable","stacktrace":"github.com/keyval-dev/odigos/odiglet/pkg/ebpf.(*InstrumentationDirector).Instrument.func1\n\t/go/src/github.com/keyval-dev/odigos/odiglet/pkg/ebpf/director.go:58"}
{"level":"info","ts":1698929256.8893397,"caller":"process/analyze.go:109","msg":"No symbols found in binary, trying to find functions using .gosymtab"}
{"level":"error","ts":1698929257.0632849,"caller":"process/analyze.go:93","msg":"Failed to find functions","error":".gosymtab section not found in target binary, make sure this is a Go application","stacktrace":"go.opentelemetry.io/auto/internal/pkg/process.(*Analyzer).Analyze\n\t/root/go/pkg/mod/go.opentelemetry.io/auto@v0.7.0-alpha/internal/pkg/process/analyze.go:93\ngo.opentelemetry.io/auto.NewInstrumentation\n\t/root/go/pkg/mod/go.opentelemetry.io/auto@v0.7.0-alpha/instrumentation.go:88\ngithub.com/keyval-dev/odigos/odiglet/pkg/ebpf.(*InstrumentationDirector).Instrument.func1\n\t/go/src/github.com/keyval-dev/odigos/odiglet/pkg/ebpf/director.go:56"}
{"level":"error","ts":1698929257.0634067,"caller":"ebpf/director.go:58","msg":"instrumentation setup failed","error":".gosymtab section not found in target binary, make sure this is a Go application","stacktrace":"github.com/keyval-dev/odigos/odiglet/pkg/ebpf.(*InstrumentationDirector).Instrument.func1\n\t/go/src/github.com/keyval-dev/odigos/odiglet/pkg/ebpf/director.go:58"}so odiglet looks at CMD or ENTRYPOINT in docker containers? you don't scan the process tree just yet ? s6 is the init manager in gitea, probably that's why it's complaining
Describe the bug
I'm trying to make odigos working with loki, tempo and prometheus - but I can't. Trying to instrument a golang app, configured loki, tempo and prometheus as sinks in odigos, but only thing I can see is this:
Tempo has no new traces, I can't see service graph, and loki has no logs corelated to traces - is there any tutorial on how to configure and debug this product?
To Reproduce
Steps to reproduce the behavior:
Desktop (please complete the following information): linux mint os
Additional context Add any other context about the problem here.