Need help in execution of the below command line to Upload packet data directly into Elastic stack. Getting "Unknown command line option" when using the pcap2json utility.
I have cloned the project on a Ubuntu 20.04 VM. and used make command to build the pcap2json utility. Let me know if anything is amiss.
root@es7:~/pcap2json# cat /home/student/ELK/http.cap | ./pcap2json --json-packet --capture-name http --output-espush --es-compress --es-host 192.168.1.248:9200
pcap2json https://www.github/fmadio/pcap2json build:Mar 31 2023 06:07:57
[--json-packet]
Write JSON Packet meta data
[--capture-name]
Unknown command line option [--capture-name]
root@es7:~/pcap2json# cat /home/student/ELK/http.cap | ./pcap2json --json-packet --output-espush --es-compress --es-host 192.168.1.248:9200 pcap2json https://www.github/fmadio/pcap2json build:Mar 31 2023 06:07:57
[--json-packet]
Write JSON Packet meta data
[--output-espush]
Unknown command line option [--output-espush]
root@es7:~/pcap2json# cat /home/student/ELK/http.cap | ./pcap2json --json-packet --es-compress --es-host 192.168.1.248:9200
pcap2json https://www.github/fmadio/pcap2json build:Mar 31 2023 06:07:57
[--json-packet]
Write JSON Packet meta data
[--es-compress]
Unknown command line option [--es-compress]
root@es7:~/pcap2json# cat /home/student/ELK/http.cap | ./pcap2json --json-packet --es-host 192.168.1.248:9200
pcap2json https://www.github/fmadio/pcap2json build:Mar 31 2023 06:07:57
[--json-packet]
Write JSON Packet meta data
[--es-host]
Unknown command line option [--es-host]
root@es7:~/pcap2json# ./pcap2json --help
pcap2json https://www.github/fmadio/pcap2json build:Mar 31 2023 06:07:57
[--help]
fmad engineering all rights reserved
http://www.fmad.io
pcap2json is a high speed PCAP meta data extraction utility
example converting a pcap to json:
cat /tmp/test.pcap | pcap2json > test.json
Command Line Arguments:
--index-name : capture name to use for ES Index data
--verbose : verbose output
--config : read from config file
--cpu-core : cpu map for core thread
--cpu-flow : cpu count and map for flow threads
--cpu-output : cpu map for output threads
--json-packet : write JSON packet data
--json-flow : write JSON flow data
Instance Info
--instance-id : instance id of this pcap2json FE
--instance-max : total number of pcap2json FE instances
Output Mode
--output-stdout : writes output to STDOUT
--output-espush : writes output directly to ES HTTP POST
--output-histogram : Enable histogram output and writes it to file
--output-buffercnt : number of output buffers (default is 64)
--output-keepalive : enable keep alive (persistent) ES connection
--output-filterpath : reduce data back from the ES cluster
--output-threadcnt : number of worker threads for ES push (default is 32)
--output-mergemin : minimum number of blocks to merge on output
--output-mergemax : maximum number of blocks to merge on output
Flow specific options
--flow-samplerate : scientific notation flow sample rate. default 100e6 (100msec)
--flow-index-depth : number of root flow index to allocate defulat 6
--flow-max : maximum number of flows (default 250e3)6
--flow-top-n : only output the top N flows
--flow-top-n-circuit : output top N flows based on specified src/dest MAC
--flow-template "" : Use a customized template for JSON output
--flow-roll-read "temp file" : Capture roll read parital snapshot to disk
--flow-roll-write "temp file" : Capture roll write parital snapshot to disk
Elastic Stack options
--es-host : Sets the ES Hostname
--es-timeout : Sets ES connection timeout in milliseconds (Default: 2000 msec)
--es-compress : enables gzip compressed POST
--es-null : use ES Null target for perf testing
--es-queue-path : ES Output queue is file backed
ICMP options
--icmp-overwrite : overwrite IP Proto info for ICMP packets
Hi,
Need help in execution of the below command line to Upload packet data directly into Elastic stack. Getting "Unknown command line option" when using the pcap2json utility.
I have cloned the project on a Ubuntu 20.04 VM. and used make command to build the pcap2json utility. Let me know if anything is amiss.
root@es7:~/pcap2json# cat /etc/os-release NAME="Ubuntu" VERSION="20.04.3 LTS (Focal Fossa)" ID=ubuntu ID_LIKE=debian PRETTY_NAME="Ubuntu 20.04.3 LTS" VERSION_ID="20.04" HOME_URL="https://www.ubuntu.com/" SUPPORT_URL="https://help.ubuntu.com/" BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/" PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy" VERSION_CODENAME=focal UBUNTU_CODENAME=focal
root@es7:~/pcap2json# cat /home/student/ELK/http.cap | ./pcap2json --json-packet --capture-name http --output-espush --es-compress --es-host 192.168.1.248:9200 pcap2json https://www.github/fmadio/pcap2json build:Mar 31 2023 06:07:57 [--json-packet] Write JSON Packet meta data [--capture-name] Unknown command line option [--capture-name] root@es7:~/pcap2json# cat /home/student/ELK/http.cap | ./pcap2json --json-packet --output-espush --es-compress --es-host 192.168.1.248:9200 pcap2json https://www.github/fmadio/pcap2json build:Mar 31 2023 06:07:57 [--json-packet] Write JSON Packet meta data [--output-espush] Unknown command line option [--output-espush] root@es7:~/pcap2json# cat /home/student/ELK/http.cap | ./pcap2json --json-packet --es-compress --es-host 192.168.1.248:9200 pcap2json https://www.github/fmadio/pcap2json build:Mar 31 2023 06:07:57 [--json-packet] Write JSON Packet meta data [--es-compress] Unknown command line option [--es-compress] root@es7:~/pcap2json# cat /home/student/ELK/http.cap | ./pcap2json --json-packet --es-host 192.168.1.248:9200 pcap2json https://www.github/fmadio/pcap2json build:Mar 31 2023 06:07:57 [--json-packet] Write JSON Packet meta data [--es-host] Unknown command line option [--es-host] root@es7:~/pcap2json# ./pcap2json --help pcap2json https://www.github/fmadio/pcap2json build:Mar 31 2023 06:07:57 [--help] fmad engineering all rights reserved http://www.fmad.io
pcap2json is a high speed PCAP meta data extraction utility
example converting a pcap to json:
cat /tmp/test.pcap | pcap2json > test.json
Command Line Arguments: --index-name : capture name to use for ES Index data
--verbose : verbose output
--config : read from config file
--cpu-core : cpu map for core thread
--cpu-flow : cpu count and map for flow threads
--cpu-output : cpu map for output threads
--json-packet : write JSON packet data --json-flow : write JSON flow data
Instance Info --instance-id : instance id of this pcap2json FE --instance-max : total number of pcap2json FE instances Output Mode --output-stdout : writes output to STDOUT --output-espush : writes output directly to ES HTTP POST --output-histogram : Enable histogram output and writes it to file
--output-buffercnt : number of output buffers (default is 64)
--output-keepalive : enable keep alive (persistent) ES connection
--output-filterpath : reduce data back from the ES cluster
--output-threadcnt : number of worker threads for ES push (default is 32)
--output-mergemin : minimum number of blocks to merge on output
--output-mergemax : maximum number of blocks to merge on output
Flow specific options --flow-samplerate : scientific notation flow sample rate. default 100e6 (100msec)
--flow-index-depth : number of root flow index to allocate defulat 6
--flow-max : maximum number of flows (default 250e3)6
--flow-top-n : only output the top N flows
--flow-top-n-circuit : output top N flows based on specified src/dest MAC
--flow-template "" : Use a customized template for JSON output
--flow-roll-read "temp file" : Capture roll read parital snapshot to disk
--flow-roll-write "temp file" : Capture roll write parital snapshot to disk
Elastic Stack options --es-host : Sets the ES Hostname
--es-timeout : Sets ES connection timeout in milliseconds (Default: 2000 msec)
--es-compress : enables gzip compressed POST
--es-null : use ES Null target for perf testing
--es-queue-path : ES Output queue is file backed
ICMP options --icmp-overwrite : overwrite IP Proto info for ICMP packets