search

sipcapture / homer

HOMER - 100% Open-Source SIP, VoIP, RTC Packet Capture & Monitoring
https://sipcapture.org
GNU Affero General Public License v3.0
1.61k stars 240 forks source link

HP_ERM support for heplify #458

Closed ChicagoJay closed 3 years ago

ChicagoJay commented 3 years ago

Trying to send traffic from a SIP gateway (CUBE router) with no available ports means using the switch to mirror the data to the heplify server. We have Aruba / HP switches here, and they use HP_ERM to forward the packets. I'd like to request HP_ERM support be added to heplify.

Thanks!

lmangani commented 3 years ago

Certified non-profit, we'll gladly try implement this feature over the next cycles.

ChicagoJay commented 3 years ago

Email sent to

Certified non-profit, we'll gladly try implement this feature over the next cycles.

Woohoo! Thanks!!!

kYroL01 commented 3 years ago

Glad to do that - Once all is done I'll let you know. Thank you

ChicagoJay commented 3 years ago

If it helps, WireShark has a dissector for HP_ERM - boundary/wireshark now at wireshark/wireshark

ChicagoJay commented 3 years ago

I have attached a pcap from a listener. The traffic comes from the switch on an arbitrary port (8383 in this case) via UDP to port 7932 on the listener.

hperm dump.zip

kYroL01 commented 3 years ago

Hi @ChicagoJay Thank you ! I'm going to check for this integration from next week - Yes, the trace will help me a lot ;) Thanks for the patience! Best Regards

ChicagoJay commented 3 years ago

Hi @kYroL01 - just checking in, since it's been a month since the last reply. Thanks!

adubovikov commented 3 years ago

@kYroL01 any updates here ?

kYroL01 commented 3 years ago

Hi @ChicagoJay apologize for my late, I'll ping you in once I finished it. Best Regards!

kYroL01 commented 3 years ago

Hi @ChicagoJay I implemented the HPERM dissection in Heplify - I need couple of days for test and send the PR, once is merged the work is done :)

ChicagoJay commented 3 years ago

Hi @ChicagoJay I implemented the HPERM dissection in Heplify - I need couple of days for test and send the PR, once is merged the work is done :)

Excellent! I can't wait to try it out!

Thanks!

ChicagoJay commented 3 years ago

Hi @ChicagoJay I implemented the HPERM dissection in Heplify - I need couple of days for test and send the PR, once is merged the work is done :)

Hi @kYroL01 - Any chance I can test this out soon? It looks like I have a free morning coming up tomorrow, and I can spend some time on it, if it's ready.

Thanks!

kYroL01 commented 3 years ago

Hi @ChicagoJay the patch is done here - https://github.com/kYroL01/heplify You can test it and give me some feedback - let me know! remember to use -bpf "port 7932" when you launch heplify Thanks a lot

ChicagoJay commented 3 years ago

@kYroL01 I ran apt update, will that grab the new patch, or do I have to manually install it? Sorry - I'm still a neophyte with open source. I installed everything with the Homer installer script. Which file do I modify to add the arguments to heplify's startup string?

I see /etc/heplify-server.toml and /lib/systemd/system/heplify-server.service - should I add something in there?

kYroL01 commented 3 years ago

Hi @ChicagoJay nothing to be sorry, will help you :) first of all heplify-server is different than heplify. heplify is the capture agent, the one who catch your traffic, dissect the HP_ERM and extract SIP to send to heplify-server (and then this traffic is shown to Homer). You need to download my patched heplify and use that, but if you don't know how to do it, we'll write you a quick procedure to help you in this process. Is it ok for you ?

Thanks Michele

ChicagoJay commented 3 years ago

@kYroL01 - I didn't realize that! Of course, it was probably 2008 before I knew Java and Javascript had NOTHING to do with each other! LOL

I'll take a look at getting heplify installed, and pray I don't break anything. I'm using Debian Buster. The procedure would be very helpful, and I am sure will save you from getting questions in the future!

Thanks!

kYroL01 commented 3 years ago

Hey @ChicagoJay I don't think you can break anything ;) In the meantime try to do a git clone from this repo https://github.com/kYroL01/heplify Basically to test you need to

If anything let us know and we'll help you! Thanks!

ChicagoJay commented 3 years ago

make all returned an error:

root@homer:/home/technician/heplify# make all
go build -ldflags "-s -w"  -o heplify *.go
/bin/sh: 1: go: not found
make: *** [Makefile:7: all] Error 127

I installed golang, and I am still getting an error:

technician@homer:~/heplify$ make all
go build -ldflags "-s -w"  -o heplify *.go
go: downloading github.com/negbie/logp v0.0.0-20190313141056-04cebff7f846
go: downloading github.com/google/gopacket v1.1.18
go: downloading golang.org/x/net v0.0.0-20200822124328-c89045814202
go: downloading golang.org/x/sys v0.0.0-20200826173525-f9321e4c35a6
go: downloading github.com/gogo/protobuf v1.3.1
go: downloading github.com/negbie/freecache v1.1.0
go: downloading github.com/segmentio/encoding v0.1.15
go: downloading github.com/cespare/xxhash v1.1.0
# github.com/google/gopacket/pcap
../goProjects/pkg/mod/github.com/google/gopacket@v1.1.18/pcap/pcap_unix.go:34:10: fatal error: pcap.h: No such file or directory
 #include <pcap.h>
          ^~~~~~~~
compilation terminated.
make: *** [Makefile:7: all] Error 2

I'll see if I can figure out where I get that...

kYroL01 commented 3 years ago

@ChicagoJay that is just missing libpcap, so apt-get install libpcap for your Debian

ChicagoJay commented 3 years ago

@kYroL01 - I thought that's what it was, but I have it installed:

technician@homer:~$ sudo apt install libpcap0.8
Reading package lists... Done
Building dependency tree
Reading state information... Done
libpcap0.8 is already the newest version (1.8.1-6).
libpcap0.8 set to manually installed.
0 upgraded, 0 newly installed, 0 to remove and 0 not upgraded.
kYroL01 commented 3 years ago

@ChicagoJay Sorry, it should be libpcap-dev

ChicagoJay commented 3 years ago

@kYroL01 OK - the make all completed, and I have a binary!

I invoked it as you specified, (sudo ./heplify -bpf "port 7932" &) and got a PID. But Homer still isn't showing any data. tcpdump shows the streams coming in from the mirror: 

technician@homer:~/heplify$ sudo tcpdump host 10.31.0.1
tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
listening on ens192, link-type EN10MB (Ethernet), capture size 262144 bytes
07:54:06.985235 IP 10.31.0.1.8383 > homer.d83.org.7932: UDP, length 226
07:54:06.992316 IP 10.31.0.1.8383 > homer.d83.org.7932: UDP, length 226
07:54:07.005244 IP 10.31.0.1.8383 > homer.d83.org.7932: UDP, length 226
07:54:07.012817 IP 10.31.0.1.8383 > homer.d83.org.7932: UDP, length 226

But nothing is showing up in the GUI.

What's the next step please?

Also - what is the recommended method of invoking heplify as a service?
Is the binary self-sufficient, so I can just put it in /usr/local/bin/ or does it need other files?

Thanks! I think we're getting close!!!

kYroL01 commented 3 years ago

@ChicagoJay probably this example can help you https://github.com/kYroL01/heplify#examples I think you just need to specify where to send traffic, so your Homer instance. Anyway yes, you just need the binary of heplify.

I'll availlable later to give you more help - in the meantime try it

ChicagoJay commented 3 years ago

I moved heplify to /usr/local/bin/ then I tried sudo heplify -hs "127.0.0.1:9060" -bpf "port 7932" & and I am still not seeing anything in Homer.

ChicagoJay commented 3 years ago

I moved heplify to /usr/local/bin/ then I tried sudo heplify -hs "127.0.0.1:9060" -bpf "port 7932" & and I am still not seeing anything in Homer.

@kYroL01 some good news: when I invoke sudo heplify -hs "127.0.0.1:9060" -bpf "port 7932" -e I get 

2021/07/01 17:08:23.930086 sniffer.go:117: INFO config.Config{Iface:(*config.InterfacesConfig)(0xc00010f4a0), Logging:(*logp.Logging)(0xc00010a0a0), Mode:"SIPRTCP", Dedup:false, Filter:"", Discard:"", DiscardMethod:"", DiscardSrcIP:"", Zip:false, HepServer:"127.0.0.1:9060", HepNodePW:"", HepNodeID:0x7d2, HepNodeName:"", Network:"udp", Protobuf:false, Reassembly:false, SendRetries:0x40, Version:false}
2021/07/01 17:08:23.930161 sniffer.go:118: INFO &config.InterfacesConfig{Device:"any", Type:"pcap", ReadFile:"", WriteFile:"", RotationTime:60, PortRange:"5060-5090", WithVlan:false, WithErspan:false, Snaplen:8192, BufferSizeMb:32, ReadSpeed:false, OneAtATime:false, Loop:1, FanoutID:0x0, FanoutWorker:4, CustomBPF:"port 7932"}
2021/07/01 17:08:23.930171 sniffer.go:119: INFO bpf: port 7932
2021/07/01 17:08:23.930177 sniffer.go:126: INFO ostype: linux, osarch: amd64
2021/07/01 17:08:24.935327 decoder.go:337: INFO Packet was successfully decoded with HPERM layer decoder.

So heplify is definitely getting the packets, recognizing them, and decoding them! Now I just need to figure out how to get them to Homer....

lmangani commented 3 years ago

@ChicagoJay

ChicagoJay commented 3 years ago

@lmangani Everything is running on a single VM, so shouldn't my command line sudo heplify -hs "127.0.0.1:9060" -bpf "port 7932" have worked?

I modified heplify.service to the following:

[Unit]
Description=Captures packets from wire and sends them to Homer
After=network.target

[Service]
ExecStart=/usr/local/bin/heplify -i any -t af_packet -hs "127.0.0.1:9060" -bpf "port 7932" -m SIPRTCP
ExecStop=/bin/kill ${MAINPID}
Restart=on-failure
RestartSec=10s
Type=simple

[Install]
WantedBy=multi-user.target

Then I did the daemon-reload, start and enable, and the service is up and running:

technician@homer:/etc/systemd/system$ ps aux | grep heplif
root       921  0.0  0.4 1163076 35112 ?       Ssl  Jun30   1:04 /usr/local/bin/heplify-server -config=/etc/heplify-server.toml
root     20020  0.6  0.9 1413852 81160 ?       Ssl  08:23   0:00 /usr/local/bin/heplify -i any -t af_packet -hs 127.0.0.1:9060 -bpf port 7932 -m SIPRTCP
technic+ 20037  0.0  0.0   6208   884 pts/0    S+   08:23   0:00 grep heplif

But I am still not seeing any data in Homer's main screen.

Do I need to enter some kind of string into the Call SIP Search area, in order to see data in the result zone, or should the results just populate, with nothing in the search?

ChicagoJay commented 3 years ago

OMG - I'm an idiot! I hit search, and I HAVE DATA!!! It looks like I just needed to reload the page. Woohoo!!!!

Thanks @lmangani & @kYroL01 - you guys ROCK!

lmangani commented 3 years ago

@ChicagoJay happy to help! if you feel like helping us back, fill this in: https://docs.google.com/forms/d/e/1FAIpQLScba0IC6KPiKb6T2AyHefJembwhPGf1q3tNgZlhbHgYAjc_Jw/viewform

ChicagoJay commented 3 years ago

@lmangani Done! I hope it helps.

kYroL01 commented 3 years ago

@ChicagoJay I'm really really glad that this work - Enjoy Homer :)) Cheers!!