search

odigos-io / opentelemetry-go-instrumentation

OpenTelemetry auto-instrumentation for Go applications
Apache License 2.0
292 stars 44 forks source link

Got panic "cant find keyval map" #39

Closed vasiliy-grinko closed 2 years ago

vasiliy-grinko commented 2 years ago

Expected Behavior

Working go-lang auto instrumentation

Actual Behavior

In sidecar I got this:

{"level":"info","ts":1666358871.3476882,"caller":"cli/main.go:22","msg":"starting Go OpenTelemetry Agent ..."}                                                                                                                                  
{"level":"info","ts":1666358871.3477695,"caller":"opentelemetry/controller.go:92","msg":"Establishing connection to OpenTelemetry collector ..."}                                                                                               │
{"level":"info","ts":1666358873.3498256,"caller":"process/discover.go:42","msg":"found process","pid":7}                                                                                                                                  
 **panic: cant find keyval map**                                                                                                                                                                                                                                                                                                                                                                                                                               
 goroutine 1 [running]:                                                                                                                                                                                                                          
 github.com/keyval-dev/opentelemetry-go-instrumentation/pkg/process.(*processAnalyzer).findKeyvalMmap(0xb51e40?, 0xc0000c0058?)                                                                                                                  │
     /app/pkg/process/analyze.go:77 +0x14f                                                                                                                                                                                                       
 github.com/keyval-dev/opentelemetry-go-instrumentation/pkg/process.(*processAnalyzer).Analyze(0xc00009e2a0?, 0x7, 0x2?)           /app/pkg/process/analyze.go:103 +0x1a5                                                                                                                                                                                   main.main()                                                                                                                                                                                                                                     /app/cli/main.go:59 +0x37f

UPDATE: Fixed panic by adding init container with odigos launcher - I don't understand why do I need add this launcher. After that error I have another error:

{"level":"info","ts":1666363476.3613198,"caller":"process/analyze.go:73","msg":"found addr of keyval map","addr":140516020199424}                                                                                                               │
{"level":"info","ts":1666363476.3766043,"caller":"process/analyze.go:136","msg":"found relevant function for instrumentation","function":"net/http.(*ServeMux).ServeHTTP","returns":2}                                                          │
{"level":"info","ts":1666363476.378454,"caller":"process/analyze.go:136","msg":"found relevant function for instrumentation","function":"google.golang.org/grpc/internal/transport.(*http2Client).createHeaderFields","returns":3}              
{"level":"info","ts":1666363476.3786323,"caller":"process/analyze.go:136","msg":"found relevant function for instrumentation","function":"google.golang.org/grpc/internal/transport.(*decodeState).decodeHeader","returns":6}                   
{"level":"info","ts":1666363476.3787887,"caller":"process/analyze.go:136","msg":"found relevant function for instrumentation","function":"google.golang.org/grpc.(*ClientConn).Invoke","returns":2}                                             │
{"level":"info","ts":1666363476.3791306,"caller":"process/analyze.go:136","msg":"found relevant function for instrumentation","function":"google.golang.org/grpc.(*Server).handleStream","returns":5}                                           │
{"level":"info","ts":1666363476.379253,"caller":"cli/main.go:64","msg":"target process analysis completed","pid":13,"go_version":"1.15.0","dependencies":{"contrib.go.opencensus.io/exporter/ocagent":"v0.6.0","github.com/beorn7/perks":"v1.0. │
1","github.com/census-instrumentation/opencensus-proto":"v0.2.1","github.com/cespare/xxhash/v2":"v2.1.1","github.com/golang/groupcache":"v0.0.0-20200121045136-8c9f03a8e57e","github.com/golang/protobuf":"v1.4.0","github.com/grpc-ecosystem/g │
o-grpc-prometheus":"v1.2.0","github.com/grpc-ecosystem/grpc-gateway":"v1.14.4","github.com/matttproud/golang_protobuf_extensions":"v1.0.1","github.com/prometheus/client_golang":"v1.6.0","github.com/prometheus/client_model":"v0.2.0","github │
.com/prometheus/common":"v0.9.1","github.com/prometheus/procfs":"v0.0.11","go.opencensus.io":"v0.22.3","golang.org/x/net":"v0.0.0-20200425230154-ff2c4b7c35a0","golang.org/x/sync":"v0.0.0-20200317015054-43a5402ce75a","golang.org/x/sys":"v0. 0.0202004300824071f5687305801","golang.org/x/text":"v0.3.2","google.golang.org/api":"v0.22.0","google.golang.org/genproto":"v0.0.020200430143042b979b6f78d84","google.golang.org/grpc":"v1.29.1","google.golang.org/protobuf":"v1.21.0"},"total_functions_found":5}                                                                                                                                                                                                                        {"level":"info","ts":1666363476.3793523,"caller":"cli/main.go:70","msg":"invoking instrumentors"}                                                                                                                                               
{"level":"info","ts":1666363476.3992898,"logger":"allocator","caller":"allocator/allocator_linux.go:19","msg":"Loading allocator","start_addr":140516020199424,"end_addr":140516032782336}                                                      │
{"level":"info","ts":1666363476.4011118,"caller":"instrumentors/runner.go:68","msg":"loading instrumentor","name":"google.golang.org/grpc"}                                                                                                     │
{"level":"info","ts":1666363476.4045897,"caller":"inject/injector.go:67","msg":"Injecting variables","vars":{"clientconn_target_ptr_pos":24,"end_addr":140516032782336,"is_registers_abi":false,"start_addr":140516020199424,"total_cpus":4}}   │
{"level":"error","ts":1666363476.4090192,"caller":"instrumentors/runner.go:71","msg":"error while loading instrumentors, cleaning up","name":"google.golang.org/grpc","error":"field UprobeHttp2ClientCreateHeaderFields: program uprobe_Http2C │
│ lient_CreateHeaderFields: load program: invalid argument: Unrecognized arg#0 type PTR\n; int uprobe_Http2Client_CreateHeaderFields(struct pt_regs *ctx)

Steps to Reproduce the Problem

  1. Using otel operator and otel collector v0.55.0 and emojivoto app for testing
  2. Add annotations instrumentation.opentelemetry.io/golang-target-exec: /usr/local/bin/emojivoto-emoji-svc instrumentation.opentelemetry.io/inject-golang: "true"

Additional Info

V1ct0rHM commented 1 year ago

I'm having this same issue, did you manage to resolve it?