search

sipcapture / rtcagent

RTCAgent is an eBPF powered HEP Agent for HOMER/HEPIC
https://sipcapture.org
GNU Affero General Public License v3.0
36 stars 2 forks source link

rtcagent w/ stripped opensips binary #9

Closed ratetel closed 4 months ago

ratetel commented 8 months ago

hello.... we have opensips installed in an lxc with deb 11. We are trying to get rtcagent running with rtcagent opensips -P9061 -S127.0.0.1

resulting in : opensips_2024/03/25 13:25:29 RTCAGENT :: version :linux_x86_64:0.0.4-20240305-048b153:[CORE] opensips_2024/03/25 13:25:29 RTCAGENT :: start to run EBPFProbeOpensips module opensips_2024/03/25 13:25:29 RTCAGENT :: opensips - nosearch: false 2024/03/25 13:25:29 HEP client will be started 2024/03/25 13:25:29 HEP client started 2024/03/25 13:25:29 RTCAGENT :: pid info :616 -127.0.0.1 opensips_2024/03/25 13:25:29 RTCAGENT :: Module.Run() opensips_2024/03/25 13:25:29 EBPFProbeOpensips BPF bytecode filename: [user/bytecode/opensips_kern.o] opensips_2024/03/25 13:25:29 EBPFProbeOpensips Opensips: 1, Version:opensips, binrayPath:/usr/sbin/opensips opensips_2024/03/25 13:25:30 couldn't start bootstrap manager error:4 errors occurred:

, probes activation validation failed

we noticed in the opensips_kern.h that in both spots the udp_send function was called, it was commented out.

any advice on what we might be doing wrong?

adubovikov commented 7 months ago

did you strip the binary ?

voransoy commented 4 months ago

Hi there I have the same issue with the rtcagent. I have installed the prebuilt binaries with dpkg. 

root@mgw:~# rtcagent opensips --hep-server=192.168.0.10 --hep-port=9060 -m /usr/sbin/opensips --hep-transport=udp
opensips_2024/07/24 16:34:53 RTCAGENT :: version :linux_x86_64:0.0.7-20240327-599486c:[CORE]
opensips_2024/07/24 16:34:53 RTCAGENT :: start to run EBPFProbeOpensips module
opensips_2024/07/24 16:34:53 RTCAGENT :: opensips - nosearch: false
2024/07/24 16:34:53 HEP client will be started
2024/07/24 16:34:53 HEP client started
2024/07/24 16:34:53 RTCAGENT :: pid info :789018 -192.168.0.10
opensips_2024/07/24 16:34:53 RTCAGENT ::    Module.Run()
opensips_2024/07/24 16:34:53 EBPFProbeOpensips  BPF bytecode filename: [user/bytecode/opensips_kern.o]
opensips_2024/07/24 16:34:53 EBPFProbeOpensips  Opensips: 1, Version:opensips, binrayPath:/usr/sbin/opensips
opensips_2024/07/24 16:34:54 couldn't start bootstrap manager error:4 errors occurred:
    * error:opening uprobe: symbol proto_udp_send: not found , isRet:false, opts:&{0 0 0 0 0}, {UID:, EbpfFuncName:msg_send_udp}
    * error:opening uprobe: symbol proto_udp_send: not found , isRet:true, opts:&{0 0 0 0 0}, {UID:, EbpfFuncName:msg_ret_send_udp}
    * error:opening uprobe: symbol proto_tcp_send: not found , isRet:false, opts:&{0 0 0 0 0}, {UID:, EbpfFuncName:msg_send_tcp}
    * error:opening uprobe: symbol proto_tcp_send: not found , isRet:true, opts:&{0 0 0 0 0}, {UID:, EbpfFuncName:msg_ret_send_tcp}

, probes activation validation failed
lmangani commented 4 months ago

@voransoy please specify the version of your kernel as well as the opensips version in use

voransoy commented 4 months ago

Hi there,

root@mgw:~# opensips -V
version: opensips 3.4.6 (x86_64/linux)
flags: STATS: On, DISABLE_NAGLE, USE_MCAST, SHM_MMAP, PKG_MALLOC, Q_MALLOC, F_MALLOC, HP_MALLOC, DBG_MALLOC, FAST_LOCK-ADAPTIVE_WAIT
ADAPTIVE_WAIT_LOOPS=1024, MAX_RECV_BUFFER_SIZE 262144, MAX_LISTEN 16, MAX_URI_SIZE 1024, BUF_SIZE 65535
poll method support: poll, epoll, sigio_rt, select.
git revision: 40206eeef
main.c compiled on  with gcc 10
root@mgw:~# uname -r
5.10.0-30-amd64
lmangani commented 4 months ago

@voransoy did you attempt building on your system or is this with the generic distribution? If so, could you please try v0.0.8

voransoy commented 4 months ago

Hi Lorenzo I haven't attempted to build it from source. I have just downloaded the v0.0.8 debian package and it is the same.

root@mgw:~# rtcagent opensips --hep-server=192.168.0.10 --hep-port=9060 -m /usr/sbin/opensips --hep-transport=udp
opensips_2024/07/24 17:13:21 RTCAGENT :: version :linux_x86_64:0.0.8-20240724-006bece:[CORE]
opensips_2024/07/24 17:13:21 RTCAGENT :: start to run EBPFProbeOpensips module
opensips_2024/07/24 17:13:21 RTCAGENT :: opensips - nosearch: false
2024/07/24 17:13:21 HEP client will be started
2024/07/24 17:13:21 HEP client started
2024/07/24 17:13:21 RTCAGENT :: pid info :789803 -192.168.0.10
opensips_2024/07/24 17:13:21 RTCAGENT ::    Module.Run()
opensips_2024/07/24 17:13:21 EBPFProbeOpensips  BPF bytecode filename: [user/bytecode/opensips_kern.o]
opensips_2024/07/24 17:13:21 EBPFProbeOpensips  Opensips: 1, Version:opensips, binrayPath:/usr/sbin/opensips
opensips_2024/07/24 17:13:21 couldn't start bootstrap manager error:4 errors occurred:
    * error:opening uprobe: symbol proto_udp_send: not found , isRet:false, opts:&{0 0 0 0 0 }, {UID:, EbpfFuncName:msg_send_udp}
    * error:opening uprobe: symbol proto_udp_send: not found , isRet:true, opts:&{0 0 0 0 0 }, {UID:, EbpfFuncName:msg_ret_send_udp}
    * error:opening uprobe: symbol proto_tcp_send: not found , isRet:false, opts:&{0 0 0 0 0 }, {UID:, EbpfFuncName:msg_send_tcp}
    * error:opening uprobe: symbol proto_tcp_send: not found , isRet:true, opts:&{0 0 0 0 0 }, {UID:, EbpfFuncName:msg_ret_send_tcp}

, probes activation validation failed

I will try to build it from the source.

lmangani commented 4 months ago

Could you confirm if your opensips binary is stripped?

voransoy commented 4 months ago

I am using prebuild binaries from the official debian repos. I am not sure but it should be stripped.

lmangani commented 4 months ago

That's most likely the root cause. If this is true, building rtcagent locally will produce the same results.

From some old post of Bogdan I see this is most likely still the case.

Only when generating packages (like debs), the debug symbols are stripped.

lmangani commented 4 months ago

Closing as we have no workaround for stripped binaries. rtcagent currently requires unstripped binaries to probe functions.