aobszyns
commented
6 years ago Same issue is here: https://github.com/shadow-box/Palo-Alto-Networks-ELK-Stack
Should be long:
The packet capture (pcap) ID is a 64 bit unsigned integral denoting an ID to correlate threat pcap files with extended pcaps taken as a part of that flow. All threat logs will contain either a pcap_id of 0 (no associated pcap), or an ID referencing the extended pcap file.
punisherVX
Please fill out the information below with as much detail as possible.
Expected behavior and actual behavior.
Logstash unable to index event to elasticsearch, when receiving threat logs (type spyware) from PA when pcap is enabled on DNS signatures. Log event never shows up in elasticsearch/kibana.
Looks like problem is related to elasticsearch index for PCAP_ID. Error message indicates integer out of range. Complete error message below:
From /var/log/logstash/logstash-plain.log [2018-07-26T12:37:19,393][WARN ][logstash.outputs.elasticsearch] Could not index event to Elasticsearch. {:status=>400, :action=>["index", {:_id=>nil, :_index=>"threat-2018.07.26", :_type=>"doc", :_routing=>nil}, #], :response=>{"index"=>{"_index"=>"threat-2018.07.26", "_type"=>"doc", "_id"=>"47cq1mQB_Rij5ncyFCCN", "status"=>400, "error"=>{"type"=>"mapper_parsing_exception", "reason"=>"failed to parse [PCAP_ID]", "caused_by"=>{"type"=>"illegal_argument_exception", "reason"=>"Value [1204318434888780917] is out of range for an integer"}}}}}
Steps to reproduce the problem.
Enable packet capture for DNS-Signatures on Anti-Spyware profile on Palo Alto. (same for both single packet and extended pcap). Generate dns sinkhole log event by doing nslookup to malicous domain, while tailing /var/log/logstash/logstash-plain.log on logstash server.
Specifications like the version of the project, operating system, or hardware.
Tested with both current master and develop branch (release 3.0) - both have same problem. Tested with PAN-OS 8.0 and 8.1 - both have same problem. Logstash version 6.2.4