search

sipcapture / captagent

100% Open-Source Packet Capture Agent for HEP
https://sipcapture.org
GNU Affero General Public License v3.0
167 stars 75 forks source link

Capture CHAN_SS7 MTPD3 #217

Closed DanielRwx closed 5 years ago

DanielRwx commented 5 years ago

Hello, i have some SS7 Gateway's using CHAN_SS7 (https://www.netfors.com/chan_ss7/) CHAN_SS7 is working fine from Brazil with SS7 Telco. MTP3D, have option to debug, (#MTP3PDUDUMP="-m /tmp/mtp3d.pcap") If i enable this debug, i can see all SS7 messages... Is possible CAPTAGENT to get output from this pcap and sent to homer?

adubovikov commented 5 years ago

You can test it by using -D flag. We have tested with SIGTRAN, so, theoreticaly you can do same for any dump, by skipping headers...

DanielRwx commented 5 years ago

You can test it by using -D flag. We have tested with SIGTRAN, so, theoreticaly you can do same for any dump, by skipping headers...

Thanks! do you have any exemple, how i can configure Captagent do read this pcap file? i only have configured using SIP in case of SIP i listen 5060 on inteface.

Thanks!

adubovikov commented 5 years ago

https://github.com/sipcapture/captagent/blob/master/conf/socket_pcap.xml#L29-L42 ->

-> https://github.com/sipcapture/captagent/blob/master/conf/captureplans/isup_capture_plan.cfg

parse_isup_to_json();

In your case you should check, what is inside your pcap, what transport it uses and after do "manipulation"

DanielRwx commented 5 years ago

@adubovikov in the socket_pcap, teoricaly i need to specify where the .cap file is right? i don't understand how socket is read the .pcap file

Thanks!!

DanielRwx commented 5 years ago

Example of .PCAP file on wireshark: MTP3D

kYroL01 commented 5 years ago

@DanielRwx No, in socket_pcap.xml you basically specify dissection, so in your case you have to enable the sctp socket in order to correctly parse your pcap. So, in your case:

<profile name="socketspcap_sctp" description="SCTP Socket" enable="true" serial="2014010402">
            <settings>
                <param name="dev" value="any"/>
                <param name="promisc" value="true"/>
                <param name="reasm" value="true"/>
                <param name="ipv4fragments" value="true"/>
                <param name="ipv6fragments" value="true"/>
                <param name="proto-type" value="sip"/>
                <param name="capture-plan" value="isup_capture_plan.cfg"/>
                <param name="filter">
                    <value>proto 132</value>
                </param>
            </settings>
        </profile>

Then, in isup_capture_plan.cfg you can specify action to be taken calling commands you can find here

and to pass the pcap to captagent just simply do this ./captagent -x 10 -D path-to-pcap. (-x 10 is to put debug log to 10) in socket_pcap.c we call pcap functions to sniff traffic from devicd or from pcap; then the pkts are sent to dissector that do the job, and return the result of dissection. This is how internally captagent works, in few words.

If u need more, just ask. Thank you

adubovikov commented 5 years ago

@kYroL01 no, @DanielRwx has "E1" stream, that only has Frames and MTP2/3 messages. @DanielRwx to read a pcap file you have to use -D flag, but in your case in the socket_pcap you should implement reader NON IP packets (at this position https://github.com/sipcapture/captagent/blob/master/src/modules/socket/pcap/socket_pcap.c#L210) and call https://github.com/sipcapture/captagent/blob/master/src/modules/socket/pcap/socket_pcap.c#L537-L538 to do direct MTP parser.

DanielRwx commented 5 years ago

Thanks, tomorow i go to build a lab with one digium E1 card and try to simulate this, and send a feedback of progress

DanielRwx commented 5 years ago

Hello @adubovikov you can help me? I'm a network/telecom enginer, i have basic programing knowlogment.

/usr/local/captagent/sbin/captagent -D mtp3d.pcap

i need help to filter and correct load pcap.

[NOTICE] Loaded core config [ERR] socket_pcap.c:848 Failed to compile filter "(portrange 5060-5091)": MTP2 link-layer type filtering not implemented [ERR] socket_pcap.c:848 Failed to compile filter "(portrange 10000-20000 and len >=20 ) and (ip and ip[6] & 0x2 = 0 and ip[6:2] & 0x1fff = 0 and udp and udp[8] & 0xc0 = 0x80 and udp[9] >= 0xc8 && udp[9] <= 0xcc)": MTP2 link-layer type filtering not implemented [ERR] socket_pcap.c:1044 Couldn't get stats on interface [(null)], index [0] [ERR] socket_pcap.c:1044 Couldn't get stats on interface [(null)], index [1] [ERR] socket_pcap.c:952 fatal: unsupported interface type [140] [140]

mtp3d.zip

DanielRwx commented 5 years ago

This is my envoriment:

SS7

adubovikov commented 5 years ago

This patch: https://github.com/sipcapture/captagent/commit/33d8a568513a232b390cf5ae908aae7dfc213a93

socket_pcap.xml should have no filter

 <profile name="socketspcap_sctp" description="SCTP Socket" enable="true" serial="2014010402">
            <settings>
                <param name="dev" value="any"/>
                <param name="promisc" value="true"/>
                <param name="reasm" value="false"/>
                <param name="ipv4fragments" value="false"/>
                <param name="ipv6fragments" value="false"/>
                <param name="proto-type" value="sip"/>
                <param name="capture-plan" value="isup_capture_plan.cfg"/>
                <param name="filter">
                    <value></value>
                </param>
            </settings>
        </profile>

/usr/local/captagent/sbin/captagent -D /tmp/mtp3d.pcap

will make this result:

{"cic":126,"msg_type":12,"msg_name":"REL","cause":{"standard_num":0,"standard_name":"ITU-T","location_num":5,"location_name":"private network serving the remote user (RPN)","itu_class_name":"normal event","itu_class_num":0,"itu_cause_name":"Normal call clearing","itu_cause_num":16},"opc":12251,"dpc":10560}

DanielRwx commented 5 years ago

Hello! Thanks, i have installed last build in a new machine...

If i try to run with the "socketspcap_sip"enable="true" serial="2014010402"> i get this output:

[root@homer captagent]# /usr/local/captagent/sbin/captagent -D /root/mtp3d.pcap [NOTICE] Loaded core config [ERR] socket_pcap.c:1243 bad values in the config [ERR] socket_pcap.c:884 Failed to compile filter "(port 5060)": MTP2 link-layer type filtering not implemented [ERR] socket_pcap.c:884 Failed to compile filter "(port 5060)": MTP2 link-layer type filtering not implemented [ERR] conf_function.c:458 find_export_record: not found parse error (37,24-25): unknown command, missing loadmodule?

If i only enable socketpcap_sctp like this:

<profile name="socketspcap_sctp" description="SCTP Socket" enable="true" serial="2014010402">
            <settings>
                <param name="dev" value="any"/>
                <param name="promisc" value="true"/>
                <param name="reasm" value="false"/>
                <param name="ipv4fragments" value="false"/>
                <param name="ipv6fragments" value="false"/>
                <param name="proto-type" value="sip"/>
                <param name="capture-plan" value="isup_capture_plan.cfg"/>
                <param name="filter">
                    <value></value>
                </param>
            </settings>

And try to run:

[root@homer captagent]# /usr/local/captagent/sbin/captagent -D /root/mtp3d.pcap [NOTICE] Loaded core config [ERR] socket_pcap.c:1243 bad values in the config ERROR: loading config file(/usr/local/captagent/etc/captagent/captureplans/isup_capture_plan.cfg): No such file or directory

I created the isup_capture_plan.cfg:

https://github.com/sipcapture/captagent/blob/master/conf/captureplans/isup_capture_plan.cfg

After, i get this output:

/usr/local/captagent/sbin/captagent -D /root/mtp3d.pcap [NOTICE] Loaded core config [ERR] socket_pcap.c:1243 bad values in the config [ERR] conf_function.c:458 find_export_record: not found parse error (4,24-25): unknown command, missing loadmodule? ERROR: bad config file (1 errors) [ERR] socket_pcap.c:1085 Couldn't get stats on interface [(null)], index [0] WARNING: run_actions: null action list (rec_level=2) WARNING: run_actions: null action list (rec_level=2) WARNING: run_actions: null action list (rec_level=2) WARNING: run_actions: null action list (rec_level=2) WARNING: run_actions: null action list (rec_level=2) WARNING: run_actions: null action list (rec_level=2) WARNING: run_actions: null action list (rec_level=2) WARNING: run_actions: null action list (rec_level=2) WARNING: run_actions: null action list (rec_level=2) WARNING: run_actions: null action list (rec_level=2) WARNING: run_actions: null action list (rec_level=2) WARNING: run_actions: null action list (rec_level=2) WARNING: run_actions: null action list (rec_level=2) WARNING: run_actions: null action list (rec_level=2) WARNING: run_actions: null action list (rec_level=2) WARNING: run_actions: null action list (rec_level=2) WARNING: run_actions: null action list (rec_level=2) WARNING: run_actions: null action list (rec_level=2) WARNING: run_actions: null action list (rec_level=2) WARNING: run_actions: null action list (rec_level=2) WARNING: run_actions: null action list (rec_level=2) WARNING: run_actions: null action list (rec_level=2) WARNING: run_actions: null action list (rec_level=2) WARNING: run_actions: null action list (rec_level=2) WARNING: run_actions: null action list (rec_level=2) WARNING: run_actions: null action list (rec_level=2) WARNING: run_actions: null action list (rec_level=2) WARNING: run_actions: null action list (rec_level=2) WARNING: run_actions: null action list (rec_level=2) WARNING: run_actions: null action list (rec_level=2) WARNING: run_actions: null action list (rec_level=2) WARNING: run_actions: null action list (rec_level=2) WARNING: run_actions: null action list (rec_level=2) WARNING: run_actions: null action list (rec_level=2) WARNING: run_actions: null action list (rec_level=2) WARNING: run_actions: null action list (rec_level=2) WARNING: run_actions: null action list (rec_level=2) WARNING: run_actions: null action list (rec_level=2) WARNING: run_actions: null action list (rec_level=2) WARNING: run_actions: null action list (rec_level=2) WARNING: run_actions: null action list (rec_level=2) WARNING: run_actions: null action list (rec_level=2) WARNING: run_actions: null action list (rec_level=2)

This is my pcap file:mtp3d.zip

Thanks in advance

adubovikov commented 5 years ago

First of all you did not read my comment property. I wrote clearly: don't set any filters. Second you did not load the module protocol_ss7

DanielRwx commented 5 years ago

Hello.. my bad! i removed all filter from socke_pcap

All is like this:

<param name="filter">
                    <value></value>

If i check captagent.xml

Have the protocol_ss7:
 <load module="protocol_ss7" register="local"/>

What is my mistake?

Thanks

adubovikov commented 5 years ago

the protocol_ss7 should be load at first position

DanielRwx commented 5 years ago

Hello! i have changed the protocol_ss7 to frist position, Now i get only a error about interface status:

I have enabled the debug:

/usr/local/captagent/sbin/captagent -D mtp3d.pcap -x 10 [NOTICE] Loaded core config [DEBUG] modules.c:50 Loading module: [protocol_ss7] [NOTICE] Loaded protocol_ss7 [DEBUG] modules.c:50 Loading module: [transport_hep] [NOTICE] Loaded transport_hep [DEBUG] modules.c:50 Loading module: [protocol_sip] [NOTICE] Loaded protocol_sip [DEBUG] modules.c:50 Loading module: [database_hash] [NOTICE] Loaded database_hash [DEBUG] modules.c:50 Loading module: [protocol_rtcp] [NOTICE] Loaded protocol_rtcp [DEBUG] modules.c:50 Loading module: [socket_pcap] [NOTICE] Loaded socket_pcap [ERR] socket_pcap.c:1243 bad values in the config [DEBUG] socket_pcap.c:801 Activating device: (null)

[NOTICE] Sending file: mtp3d.pcap [NOTICE] Using filter: [DEBUG] conf_function.c:453 find_export_record: found in module protocol_ss7 [/usr/local/captagent/lib/captagent/modules] [DEBUG] conf_function.c:453 find_export_record: found in module transport_hep [/usr/local/captagent/lib/captagent/modules] [DEBUG] conf_function.c:453 find_export_record: found in module protocol_sip [/usr/local/captagent/lib/captagent/modules] [DEBUG] captagent.c:351 The Captagent is ready [DEBUG] socket_pcap.c:997 Link offset interface type [140] [0] [DEBUG] socket_pcap.c:1046 STARTING STATS.... [ERR] socket_pcap.c:1085 Couldn't get stats on interface [(null)], index [0] [DEBUG] socket_pcap.c:1005 loop stopped by EOF [DEBUG] socket_pcap.c:1030 exit loop [DEBUG] socket_pcap.c:1034 Process, pid=11566

[DEBUG] captagent.c:88 The agent has been terminated [NOTICE] unloaded module socket_pcap [NOTICE] unloaded module protocol_rtcp [NOTICE] unloaded module database_hash [NOTICE] unloaded module protocol_sip

Thanks!

kYroL01 commented 5 years ago

Hi @DanielRwx . I think you have to change dev param with your interface instead any. Open socket_pcap.xml and change the value - I.e. if my interface is eth0, do <param name="dev" value="eth0"/> change in every socket you enabled.

p.s. if u don't know, run captagent -a that list u all the available devices on your system.

Thank you

adubovikov commented 5 years ago

this has nothing to do to the file. Just check if you sent something on the HEP port to your Homer.

On Wed, 25 Sep 2019, 04:13 DanielRwx notifications@github.com wrote:

Hello! i have changed the protocol_ss7 to frist position, Now i get only a error about interface status:

I have enabled the debug:

/usr/local/captagent/sbin/captagent -D mtp3d.pcap -x 10 [NOTICE] Loaded core config [DEBUG] modules.c:50 Loading module: [protocol_ss7] [NOTICE] Loaded protocol_ss7 [DEBUG] modules.c:50 Loading module: [transport_hep] [NOTICE] Loaded transport_hep [DEBUG] modules.c:50 Loading module: [protocol_sip] [NOTICE] Loaded protocol_sip [DEBUG] modules.c:50 Loading module: [database_hash] [NOTICE] Loaded database_hash [DEBUG] modules.c:50 Loading module: [protocol_rtcp] [NOTICE] Loaded protocol_rtcp [DEBUG] modules.c:50 Loading module: [socket_pcap] [NOTICE] Loaded socket_pcap [ERR] socket_pcap.c:1243 bad values in the config [DEBUG] socket_pcap.c:801 Activating device: (null)

[NOTICE] Sending file: mtp3d.pcap [NOTICE] Using filter: [DEBUG] conf_function.c:453 find_export_record: found in module protocol_ss7 [/usr/local/captagent/lib/captagent/modules] [DEBUG] conf_function.c:453 find_export_record: found in module transport_hep [/usr/local/captagent/lib/captagent/modules] [DEBUG] conf_function.c:453 find_export_record: found in module protocol_sip [/usr/local/captagent/lib/captagent/modules] [DEBUG] captagent.c:351 The Captagent is ready [DEBUG] socket_pcap.c:997 Link offset interface type [140] [0] [DEBUG] socket_pcap.c:1046 STARTING STATS.... [ERR] socket_pcap.c:1085 Couldn't get stats on interface [(null)], index [0] [DEBUG] socket_pcap.c:1005 loop stopped by EOF [DEBUG] socket_pcap.c:1030 exit loop [DEBUG] socket_pcap.c:1034 Process, pid=11566

[DEBUG] captagent.c:88 The agent has been terminated [NOTICE] unloaded module socket_pcap [NOTICE] unloaded module protocol_rtcp [NOTICE] unloaded module database_hash [NOTICE] unloaded module protocol_sip

Thanks!

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/sipcapture/captagent/issues/217?email_source=notifications&email_token=ABCN2JMHHHHMM5NPT7HEFULQLLCNLA5CNFSM4IXGPHHKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOD7QLLWI#issuecomment-534820313, or mute the thread https://github.com/notifications/unsubscribe-auth/ABCN2JOMK3MQZTSQOKRELCLQLLCNLANCNFSM4IXGPHHA .

adubovikov commented 5 years ago

https://github.com/sipcapture/captagent/commit/52d0ca2504f64b91be1363a13fe5e2499a0ed413

this patch turns stats to FALSE by default. If you wanna enable it, in the socket_pcap.xml set stats-enable to TRUE

but again, this has nothing to do with your MTP2 messages. It has been parsed and sent to HOMER. Please check if you have received somehitng on the HEP port (9060 or 9063)

adubovikov commented 5 years ago

Any updates ?

DanielRwx commented 5 years ago

Hello @adubovikov and @kYroL01, sorry for delay, i make a new pacap, make some calls, now no have errors and send to homer machine, now i studying now kamailio is going to undertand and put it on db.

image

Thank's a lot for help!

DanielRwx commented 5 years ago

Probably this is the problem:

-- Logs begin at Seg 2019-09-23 22:38:22 -03, end at Qui 2019-09-26 23:47:23 -03. -- Set 26 23:47:23 homer.localdomain homer[20383]: ERROR: sipcapture [hep.c:444]: parsing_hepv3_message(): NO IP's set Set 26 23:47:23 homer.localdomain homer[20384]: ERROR: sipcapture [hep.c:444]: parsing_hepv3_message(): NO IP's set Set 26 23:47:23 homer.localdomain homer[20386]: ERROR: sipcapture [hep.c:444]: parsing_hepv3_message(): NO IP's set Set 26 23:47:23 homer.localdomain homer[20384]: ERROR: sipcapture [hep.c:444]: parsing_hepv3_message(): NO IP's set Set 26 23:47:23 homer.localdomain homer[20386]: ERROR: sipcapture [hep.c:444]: parsing_hepv3_message(): NO IP's set Set 26 23:47:23 homer.localdomain homer[20386]: ERROR: sipcapture [hep.c:444]: parsing_hepv3_message(): NO IP's set Set 26 23:47:23 homer.localdomain homer[20383]: ERROR: sipcapture [hep.c:444]: parsing_hepv3_message(): NO IP's set Set 26 23:47:23 homer.localdomain homer[20384]: ERROR: sipcapture [hep.c:444]: parsing_hepv3_message(): NO IP's set Set 26 23:47:23 homer.localdomain homer[20384]: ERROR: sipcapture [hep.c:444]: parsing_hepv3_message(): NO IP's set Set 26 23:47:23 homer.localdomain homer[20381]: ERROR: sipcapture [hep.c:444]: parsing_hepv3_message(): NO IP's set Set 26 23:47:23 homer.localdomain homer[20386]: ERROR: sipcapture [hep.c:444]: parsing_hepv3_message(): NO IP's set

adubovikov commented 5 years ago

please use heplify-server, and not kamailio

On Fri, 27 Sep 2019 at 04:49, DanielRwx notifications@github.com wrote:

Probably this is the problem:

-- Logs begin at Seg 2019-09-23 22:38:22 -03, end at Qui 2019-09-26 23:47:23 -03. -- Set 26 23:47:23 homer.localdomain homer[20383]: ERROR: sipcapture [hep.c:444]: parsing_hepv3_message(): NO IP's set Set 26 23:47:23 homer.localdomain homer[20384]: ERROR: sipcapture [hep.c:444]: parsing_hepv3_message(): NO IP's set Set 26 23:47:23 homer.localdomain homer[20386]: ERROR: sipcapture [hep.c:444]: parsing_hepv3_message(): NO IP's set Set 26 23:47:23 homer.localdomain homer[20384]: ERROR: sipcapture [hep.c:444]: parsing_hepv3_message(): NO IP's set Set 26 23:47:23 homer.localdomain homer[20386]: ERROR: sipcapture [hep.c:444]: parsing_hepv3_message(): NO IP's set Set 26 23:47:23 homer.localdomain homer[20386]: ERROR: sipcapture [hep.c:444]: parsing_hepv3_message(): NO IP's set Set 26 23:47:23 homer.localdomain homer[20383]: ERROR: sipcapture [hep.c:444]: parsing_hepv3_message(): NO IP's set Set 26 23:47:23 homer.localdomain homer[20384]: ERROR: sipcapture [hep.c:444]: parsing_hepv3_message(): NO IP's set Set 26 23:47:23 homer.localdomain homer[20384]: ERROR: sipcapture [hep.c:444]: parsing_hepv3_message(): NO IP's set Set 26 23:47:23 homer.localdomain homer[20381]: ERROR: sipcapture [hep.c:444]: parsing_hepv3_message(): NO IP's set Set 26 23:47:23 homer.localdomain homer[20386]: ERROR: sipcapture [hep.c:444]: parsing_hepv3_message(): NO IP's set

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/sipcapture/captagent/issues/217?email_source=notifications&email_token=ABCN2JILSDED3NCL7OZ6RYTQLVYB3A5CNFSM4IXGPHHKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOD7XRVFA#issuecomment-535763604, or mute the thread https://github.com/notifications/unsubscribe-auth/ABCN2JPEL7K67WXA2BPNKP3QLVYB3ANCNFSM4IXGPHHA .

adubovikov commented 5 years ago

sorry but this is different topic. I will close the issue. Please open a new one if u have some issues to store date into DB.