grafana / beyla

eBPF-based autoinstrumentation of web applications and network metrics
https://grafana.com/oss/beyla-ebpf/
Apache License 2.0
1.34k stars 96 forks source link

Beyla crashing on R-lang DWARF parsing #625

Open mariomac opened 7 months ago

mariomac commented 7 months ago

Happens from 1.0 to 1.2.1. The instrumented executable is not publicly available.

eer_addr_pos:{} grpc_peer_localaddr_pos:{} grpc_st_localaddr_ptr_pos:{} grpc_st_peer_ptr_pos:{} grpc_st_remoteaddr_ptr_pos:{} grpc_status_code_ptr_pos:{} grpc_status_s_pos:{} grpc_stream_ctx_ptr_pos:{} grpc_stream_method_ptr_pos:{} grpc_stream_st_ptr_pos:{} host_ptr_pos:{} hpack_encoder_w_pos:{} http2_client_next_id_pos:{} io_writer_buf_ptr_pos:{} io_writer_n_pos:{} method_ptr_pos:{} path_ptr_pos:{} remoteaddr_ptr_pos:{} req_header_ptr_pos:{} resp_req_pos:{} status_code_ptr_pos:{} status_ptr_pos:{} tcp_addr_ip_ptr_pos:{} tcp_addr_port_ptr_pos:{} url_ptr_pos:{} value_context_val_ptr_pos:{}]"
time=2024-02-15T08:26:40.829Z level=DEBUG msg="inspecting fields for struct type" component=goexec.structMemberOffsets function=structMemberOffsetsFromDwarf type=net.TCPAddr
time=2024-02-15T08:26:40.829Z level=DEBUG msg="found struct member offset" component=goexec.structMemberOffsets const=tcp_addr_ip_ptr_pos offset="#\x00"
panic: interface conversion: interface {} is []uint8, not int64

goroutine 37 [running]:
github.com/grafana/beyla/pkg/internal/goexec.readMembers(0xc000a36ba0?, 0x1fbbd80?, 0x30f86c0?, 0xfffffffffffffffc?)
    /opt/app-root/pkg/internal/goexec/structmembers.go:272 +0x491
github.com/grafana/beyla/pkg/internal/goexec.structMemberOffsetsFromDwarf(0xc001832900)
    /opt/app-root/pkg/internal/goexec/structmembers.go:241 +0x447
github.com/grafana/beyla/pkg/internal/goexec.structMemberOffsets(0xc000cc0aa0?)
    /opt/app-root/pkg/internal/goexec/structmembers.go:145 +0x39
github.com/grafana/beyla/pkg/internal/goexec.InspectOffsets(0xc000203830, {0xc000508100?, 0x30f86c0?, 0xfffffffffffffffc?})
    /opt/app-root/pkg/internal/goexec/offsets.go:40 +0xb2
github.com/grafana/beyla/pkg/internal/discover.(*typer).inspectOffsets(0xc00017ec00, 0xc000203830)
    /opt/app-root/pkg/internal/discover/typer.go:152 +0x26d
github.com/grafana/beyla/pkg/internal/discover.(*typer).asInstrumentable(0xc00017ec00, 0xc000203830)
    /opt/app-root/pkg/internal/discover/typer.go:110 +0x17a
github.com/grafana/beyla/pkg/internal/discover.(*typer).FilterClassify(0xc00017ec00, {0xc000fbd400, 0x53, 0x0?})
    /opt/app-root/pkg/internal/discover/typer.go:94 +0x571
github.com/grafana/beyla/pkg/internal/discover.ExecTyperProvider.func1(0x0?, 0x0?)
    /opt/app-root/pkg/internal/discover/typer.go:49 +0x5e
github.com/mariomac/pipes/pkg/node.(*Middle[...]).start.func1()
    /opt/app-root/vendor/github.com/mariomac/pipes/pkg/node/node.go:185 +0x3b
created by github.com/mariomac/pipes/pkg/node.(*Middle[...]).start in goroutine 52
    /opt/app-root/vendor/github.com/mariomac/pipes/pkg/node/node.go:184 +0x279
mariomac commented 7 months ago

The issue is that we are expecting an integer number as offset and we got in memory a byte slice offset="#\x00"

According to the DWARF specification (page 227) this value can be:

mariomac commented 7 months ago

The user reporting the issue commented:

The app getting traced is an app written in R so is it possible that the app is detected as a go app ?

grcevski commented 7 months ago

Yeah I think maybe we should do some sanity checks for language detection before we try to read the dwarf, or we can just fix this in the offset scanner. It would be hard to reproduce I think, but maybe we can make sure it works with unit tests.