search

jasonish / py-idstools

idstools: Snort and Suricata Rule and Event Utilities in Python (Including a Rule Update Tool)
Other
275 stars 84 forks source link

Error eve2pcap not work #66

Open Canon88 opened 6 years ago

Canon88 commented 6 years ago

When I use eve2pcap, it does not work and throws the following exception.

[root@localhost bin]# python3 idstools-eve2pcap /tmp/eve.json --payload -o /tmp/tmp_pcap
b''
Traceback (most recent call last):
  File "idstools-eve2pcap", line 12, in <module>
    sys.exit(main())
  File "/usr/local/python3/lib/python3.6/site-packages/idstools/scripts/eve2pcap.py", line 227, in main
    dumper = pcap.dump_open(args.output)
  File "/usr/local/python3/lib/python3.6/site-packages/idstools/scripts/eve2pcap.py", line 116, in dump_open
    raise Exception(pcap_errbuf.value)
Exception: b''
jasonish commented 6 years ago

I justed tested with Python 3.6 and it works here, however, I have found that the interaction between Python, libpcap and scapy to be somewhat fragile given what version of everything is installed.

If you can provide the following, I'll look further.

I did notice that this same error is thrown when the output file is not writable by the user, rather than giving a pretty error message. Can you check that /tmp/tmp_pcap does not exist, or is at least writeablt by you. Also make sure its not a directory.

Canon88 commented 6 years ago

OS: Linux localhost.localdomain 3.10.0-693.11.6.el7.x86_64 #1 SMP Thu Jan 4 01:06:37 UTC 2018 x86_64 x86_64 x86_64 GNU/Linux Version: 0.6.3 pip3 install idstools

Yes, / tmp / tmp_pcap does not exist. I am running as root and have permission to write or create this file.

xuh881026@foxmail.com

From: Jason Ish Date: 2018-01-30 20:55 To: jasonish/py-idstools CC: Canon88; Author Subject: Re: [jasonish/py-idstools] Error eve2pcap not work (#66) I justed tested with Python 3.6 and it works here, however, I have found that the interaction between Python, libpcap and scapy to be somewhat fragile given what version of everything is installed. If you can provide the following, I'll look further. OS, if Linux the distribution. Version of idstools. Idstools installed manually or with package manager I did notice that this same error is thrown when the output file is not writable by the user, rather than giving a pretty error message. Can you check that /tmp/tmp_pcap does not exist, or is at least writeablt by you. Also make sure its not a directory. — You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub, or mute the thread.

jasonish commented 6 years ago

I've made a few fixes with respect to Python 3. Would you be willing to test?

pip3 install --upgrade https://github.com/jasonish/py-idstools/archive/master.zip

Canon88 commented 6 years ago

Yes, it is my pleasure, but my internet speed is very slow, so I'll give you feedback when I download it.

xuh881026@foxmail.com

From: Jason Ish Date: 2018-01-30 21:40 To: jasonish/py-idstools CC: Canon88; Author Subject: Re: [jasonish/py-idstools] Error eve2pcap not work (#66) I've made a few fixes with respect to Python 3. Would you be willing to test? pip3 install --upgrade https://github.com/jasonish/py-idstools/archive/master.zip — You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub, or mute the thread.

Canon88 commented 6 years ago

It seems that there is no error, but I use wireshark to open the file, do not see the inside of the packet. WireShark:

xuh881026@foxmail.com

From: Jason Ish Date: 2018-01-30 21:40 To: jasonish/py-idstools CC: Canon88; Author Subject: Re: [jasonish/py-idstools] Error eve2pcap not work (#66) I've made a few fixes with respect to Python 3. Would you be willing to test? pip3 install --upgrade https://github.com/jasonish/py-idstools/archive/master.zip — You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub, or mute the thread.

jasonish commented 6 years ago

Check the size of the pcap. Run it through tcpdump and see if there is anything: "tcpdump -r filename".

Also make sure your input eve.json file has "packet" and/or "payload" fields.

Canon88 commented 6 years ago

You can see the following output:

Will my eve.json file contain flow, http, alert and other types of data, will it affect?

This is my alert data, including the payload { "timestamp": "2018-01-26T16:27:06.051714+0800", "flow_id": 1533431743420566, "in_iface": "eth0", "event_type": "alert", "src_ip": "192.168.199.2", "src_port": 54712, "dest_ip": "58.216.107.161", "dest_port": 80, "proto": "TCP", "tx_id": 0, "alert": { "action": "allowed", "gid": 1, "signature_id": 2011719, "rev": 8, "signature": "ET POLICY Win32/Sogou User-Agent (SOGOU_UPDATER)", "category": "A Network Trojan was detected", "severity": 1 }, "http": { "hostname": "dl.pinyin.sogou.com", "url": "/patch_files_6.5/patch_1517130311_4.scel.gz?v=8.6.0.1467&h=733941BCC69A8F3E3D5A62BEBB4A2546", "http_user_agent": "SOGOU_UPDATER", "http_content_type": "application/octet-stream", "http_method": "GET", "protocol": "HTTP/1.1", "status": 200, "length": 1011 }, "app_proto": "http", "flow": { "pkts_toserver": 4, "pkts_toclient": 3, "bytes_toserver": 545, "bytes_toclient": 1620, "start": "2018-01-26T16:27:06.039062+0800" }, "payload_printable": "GET /patch_files_6.5/patch_1517130311_4.scel.gz?v=8.6.0.1467&h=733941BCC69A8F3E3D5A62BEBB4A2546 HTTP/1.1 User-Agent: SOGOU_UPDATER Host: dl.pinyin.sogou.com Cache-Control: no-cache Cookie: YYID=733941BCC69A8F3E3D5A62BEBB4A2546; IPLOC=CN3201; IMEVER=8.6.0.1467; SUV=00D7735172DE9567594000BA83C7C613

", "stream": 1 }

xuh881026@foxmail.com

From: Jason Ish Date: 2018-01-30 22:14 To: jasonish/py-idstools CC: Canon88; Author Subject: Re: [jasonish/py-idstools] Error eve2pcap not work (#66) Check the size of the pcap. Run it through tcpdump and see if there is anything: "tcpdump -r ". Also make sure your input eve.json file has "packet" and/or "payload" fields. — You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub, or mute the thread.

jasonish commented 6 years ago

You’ll need to enable the payload option in your Suricata config. Looks like you have payload_printable enabled which eve2pcap doesn’t handle. Converting it to pcap wouldn’t provide any extra value anyways, as you can see the data in it’s printable form already.