search

vipinpv85 / DPDK-Suricata_3.0

add dpdk interface and packet processing to suricata in worker mode
https://github.com/vipinpv85/DPDK-Suricata_3.0
GNU Lesser General Public License v3.0
62 stars 34 forks source link

Due to current environment, not able to recieve packet #9

Closed DubheStar closed 5 years ago

DubheStar commented 5 years ago

Hi~ :

An error occurred while I was running DPDK-suricata_3.0 root@ubuntu:~# suricata --dpdkintel EAL: Detected 4 lcore(s) EAL: No free hugepages reported in hugepages-1048576kB EAL: Probing VFIO support... EAL: PCI device 0000:02:00.0 on NUMA socket -1 EAL: Invalid NUMA socket, default to 0 EAL: probe driver: 8086:100f net_e1000_em EAL: PCI device 0000:02:04.0 on NUMA socket -1 EAL: Invalid NUMA socket, default to 0 EAL: probe driver: 8086:100f net_e1000_em EAL: PCI device 0000:02:05.0 on NUMA socket -1 EAL: Invalid NUMA socket, default to 0 EAL: probe driver: 8086:100f net_e1000_em 9/4/2019 -- 00:01:47 - - This is Suricata version 3.0 RELEASE ERROR: No interface found for DPDK Inte

root@ubuntu:~# suricata --list-dpdkintel-ports EAL: Detected 4 lcore(s) EAL: No free hugepages reported in hugepages-1048576kB EAL: Probing VFIO support... EAL: PCI device 0000:02:00.0 on NUMA socket -1 EAL: Invalid NUMA socket, default to 0 EAL: probe driver: 8086:100f net_e1000_em EAL: PCI device 0000:02:04.0 on NUMA socket -1 EAL: Invalid NUMA socket, default to 0 EAL: probe driver: 8086:100f net_e1000_em EAL: PCI device 0000:02:05.0 on NUMA socket -1 EAL: Invalid NUMA socket, default to 0 EAL: probe driver: 8086:100f net_e1000_em

--- DPDK Intel Ports ---

Details (please complete the following information):

Network devices using kernel driver

0000:02:00.0 '82545EM Gigabit Ethernet Controller (Copper) 100f' if=ens32 drv=e1000 unused=igb_uio Active

Other Network devices

Looking forward to your reply:)
vipinpv85 commented 5 years ago

Did you check yaml file ?

vipinpv85 commented 5 years ago

dpdkintel support

dpdkintel:

inputs:

DubheStar commented 5 years ago

Hi~: I runed suricata ,An error occurred again.

9/4/2019 -- 02:19:06 - - DPDK Version: DPDK 17.11.3 9/4/2019 -- 02:19:06 - - ----- Global DPDK-INTEL Config ----- 9/4/2019 -- 02:19:06 - - Number Of Ports : 2 9/4/2019 -- 02:19:06 - - Operation Mode : IPS 9/4/2019 -- 02:19:06 - - Port:0, Map:0 9/4/2019 -- 02:19:06 - - Port:0, Map:0 9/4/2019 -- 02:19:06 - - ------------------------------------ 9/4/2019 -- 02:19:06 - - ----- Match Pattern ---- 9/4/2019 -- 02:19:06 - - http: 0 9/4/2019 -- 02:19:06 - - ftp: 0 9/4/2019 -- 02:19:06 - - tls: 0 9/4/2019 -- 02:19:06 - - dns: 0 9/4/2019 -- 02:19:06 - - smtp: 0 9/4/2019 -- 02:19:06 - - ssh: 0 9/4/2019 -- 02:19:06 - - smb: 0 9/4/2019 -- 02:19:06 - - smb2: 0 9/4/2019 -- 02:19:06 - - dcerpc:0 9/4/2019 -- 02:19:06 - - tcp: 0 9/4/2019 -- 02:19:06 - - udp: 0 9/4/2019 -- 02:19:06 - - sctp: 0 9/4/2019 -- 02:19:06 - - icmpv6:0 9/4/2019 -- 02:19:06 - - gre: 0 9/4/2019 -- 02:19:06 - - raw: 0 9/4/2019 -- 02:19:06 - - ipv4: 0 9/4/2019 -- 02:19:06 - - * ipv6: 0 9/4/2019 -- 02:19:06 - - ----------------------- 9/4/2019 -- 02:19:06 - - [ERRCODE: SC_ERR_UNKNOWN_RUN_MODE(125)] - Unknown runtime mode. Aborting

I guess yaml file have problemsWould you send a yaml file to wyz05170517@gmail.com?

I will be very grateful to you;）

vipinpv85 commented 5 years ago

It looks like when you rebuild the suricata suricata.yaml.in template replaced the orginal suricata.yaml. Hence follow these steps and let me know.

  1. Open suricata.yaml and check for 'dpdkintel:'. If you do not find it then suricata.yaml is replaced. Solution> copy the suricata.yaml in github and replace in your folder

  2. run './suricata --list-runmodes', if dpdk mode is not listed Solution> please check configure.log to check if Suricata is build with dpdk option.

note: I am not sure why you want me 'I guess yaml file have problemsWould you send a yaml file to wyz05170517@gmail.com?'

please update asap

DubheStar commented 5 years ago

Yestoday,i I found the file was overwritten.Then i copy suricata.yaml file .The first error was resolved : ) This is my running record for today

root@ubuntu:~/suricata-3.0/src# ./suricata --list-runmodes EAL: Detected 4 lcore(s) EAL: No free hugepages reported in hugepages-1048576kB EAL: Probing VFIO support... EAL: PCI device 0000:02:00.0 on NUMA socket -1 EAL: Invalid NUMA socket, default to 0 EAL: probe driver: 8086:100f net_e1000_em EAL: PCI device 0000:02:04.0 on NUMA socket -1 EAL: Invalid NUMA socket, default to 0 EAL: probe driver: 8086:100f net_e1000_em EAL: PCI device 0000:02:05.0 on NUMA socket -1 EAL: Invalid NUMA socket, default to 0 EAL: probe driver: 8086:100f net_e1000_em EAL: PCI device 0000:02:06.0 on NUMA socket -1 EAL: Invalid NUMA socket, default to 0 EAL: probe driver: 8086:100f net_e1000_em EAL: PCI device 0000:02:07.0 on NUMA socket -1 EAL: Invalid NUMA socket, default to 0 EAL: probe driver: 8086:100f net_e1000_em ------------------------------------- Runmodes ------------------------------------------ RunMode Type Custom Mode Description
PCAP_DEV single Single threaded pcap live mode
---------------------------------------------------------------------
autofp Multi threaded pcap live mode. Packets from each flow are assigned to a single detect thread, unlike "pcap_live_auto" where packets from the same flow can be processed by any detect thread
---------------------------------------------------------------------
workers Workers pcap live mode, each thread does all tasks from acquisition to logging
----------------------------------------------------------------------------------------
PCAP_FILE single Single threaded pcap file mode
---------------------------------------------------------------------
autofp Multi threaded pcap file mode. Packets from each flow are assigned to a single detect thread, unlike "pcap-file-auto" where packets from the same flow can be processed by any detect thread
----------------------------------------------------------------------------------------
PFRING(DISABLED) autofp Multi threaded pfring mode. Packets from each flow are assigned to a single detect thread, unlike "pfring_auto" where packets from the same flow can be processed by any detect thread
---------------------------------------------------------------------
single Single threaded pfring mode
---------------------------------------------------------------------
workers Workers pfring mode, each thread does all tasks from acquisition to logging
----------------------------------------------------------------------------------------
NFQ autofp Multi threaded NFQ IPS mode with respect to flow
---------------------------------------------------------------------
workers Multi queue NFQ IPS mode with one thread per queue
----------------------------------------------------------------------------------------
NFLOG autofp Multi threaded nflog mode
---------------------------------------------------------------------
single Single threaded nflog mode
---------------------------------------------------------------------
workers Workers nflog mode
----------------------------------------------------------------------------------------
IPFW autofp Multi threaded IPFW IPS mode with respect to flow
---------------------------------------------------------------------
workers Multi queue IPFW IPS mode with one thread per queue
----------------------------------------------------------------------------------------
ERF_FILE single Single threaded ERF file mode
---------------------------------------------------------------------
autofp Multi threaded ERF file mode. Packets from each flow are assigned to a single detect thread
----------------------------------------------------------------------------------------
ERF_DAG autofp Multi threaded DAG mode. Packets from each flow are assigned to a single detect thread, unlike "dag_auto" where packets from the same flow can be processed by any detect thread
---------------------------------------------------------------------
single Singled threaded DAG mode
---------------------------------------------------------------------
workers Workers DAG mode, each thread does all tasks from acquisition to logging
----------------------------------------------------------------------------------------
AF_PACKET_DEV single Single threaded af-packet mode
---------------------------------------------------------------------
workers Workers af-packet mode, each thread does all tasks from acquisition to logging
---------------------------------------------------------------------
autofp Multi socket AF_PACKET mode. Packets from each flow are assigned to a single detect thread.
----------------------------------------------------------------------------------------
NETMAP(DISABLED) single Single threaded netmap mode
---------------------------------------------------------------------
workers Workers netmap mode, each thread does all tasks from acquisition to logging
---------------------------------------------------------------------
autofp Multi threaded netmap mode. Packets from each flow are assigned to a single detect thread.
----------------------------------------------------------------------------------------
UNIX_SOCKET single Unix socket mode
----------------------------------------------------------------------------------------

And this is my compile command ./configure --enable-dpdkintel --prefix=/usr --sysconfdir=/etc --localstatedir=/var

vipinpv85 commented 5 years ago

updated runmodes.c and runodes.h. Do a git pull and try.

DubheStar commented 5 years ago

HI ~

Thank you for your patience. The last problem has been solved

This is my running record rightnow,Why Unknown speed for 0? please see below detail. suricata -c /etc/suricata/suricata.yaml --dpdkintel 10/4/2019 -- 13:23:10 - - This is Suricata version 3.0 RELEASE 10/4/2019 -- 13:23:17 - - [ERRCODE: SC_ERR_DPDKINTEL_CONFIG_FAILED(275)] - No Mapping found for Port: 1 10/4/2019 -- 13:23:17 - - DPDK Version: DPDK 17.11.3 10/4/2019 -- 13:23:17 - - ----- Global DPDK-INTEL Config ----- 10/4/2019 -- 13:23:17 - - Number Of Ports : 2 10/4/2019 -- 13:23:17 - - Operation Mode : IDS 10/4/2019 -- 13:23:17 - - Port:0, Map:0 10/4/2019 -- 13:23:17 - - Port:1, Map:0 10/4/2019 -- 13:23:17 - - ------------------------------------ 10/4/2019 -- 13:23:17 - - ----- Match Pattern ---- 10/4/2019 -- 13:23:17 - - http: 0 10/4/2019 -- 13:23:17 - - ftp: 0 10/4/2019 -- 13:23:17 - - tls: 0 10/4/2019 -- 13:23:17 - - dns: 0 10/4/2019 -- 13:23:17 - - smtp: 0 10/4/2019 -- 13:23:17 - - ssh: 0 10/4/2019 -- 13:23:17 - - smb: 0 10/4/2019 -- 13:23:17 - - smb2: 0 10/4/2019 -- 13:23:17 - - dcerpc:0 10/4/2019 -- 13:23:17 - - tcp: 0 10/4/2019 -- 13:23:17 - - udp: 0 10/4/2019 -- 13:23:17 - - sctp: 0 10/4/2019 -- 13:23:17 - - icmpv6:0 10/4/2019 -- 13:23:17 - - gre: 0 10/4/2019 -- 13:23:17 - - raw: 0 10/4/2019 -- 13:23:17 - - ipv4: 0 10/4/2019 -- 13:23:17 - - * ipv6: 0 10/4/2019 -- 13:23:17 - - ----------------------- 10/4/2019 -- 13:23:17 - - all 1 packet processing threads, 4 management threads initialized, engine started. 10/4/2019 -- 13:23:17 - - [ERRCODE: SC_ERR_DPDKINTEL_CONFIG_FAILED(275)] - Unknown speed for 0

vipinpv85 commented 5 years ago
  1. did you try checking the port speed using l2fwd or testpmd?
  2. the application is build with intel NIC (check configure). If you do need non intel NIC please add in configure
DubheStar commented 5 years ago

My environment is Vmware virtual machine OS :ubuntu 18.04LTS NIC: Network devices using DPDK-compatible driver

0000:02:04.0 '82545EM Gigabit Ethernet Controller (Copper) 100f' drv=igb_uio unused=e1000 0000:02:05.0 '82545EM Gigabit Ethernet Controller (Copper) 100f' drv=igb_uio unused=e1000

1.l2fwd Port statistics ==================================== Statistics for port 0 ------------------------------ Packets sent: 3165 Packets received: 3163 Packets dropped: 0 Statistics for port 1 ------------------------------ Packets sent: 3163 Packets received: 3166 Packets dropped: 0 Aggregate statistics =============================== Total packets sent: 6328 Total packets received: 6329 Total packets dropped: 0

2.testpmd

testpmd> start io packet forwarding - ports=2 - cores=1 - streams=2 - NUMA support enabled, MP over anonymous pages disabled Logical Core 1 (socket 0) forwards packets on 2 streams: RX P=0/Q=0 (socket 0) -> TX P=1/Q=0 (socket 0) peer=02:00:00:00:00:01 RX P=1/Q=0 (socket 0) -> TX P=0/Q=0 (socket 0) peer=02:00:00:00:00:00

io packet forwarding packets/burst=32 nb forwarding cores=1 - nb forwarding ports=2 port 0: CRC stripping enabled RX queues=1 - RX desc=128 - RX free threshold=0 RX threshold registers: pthresh=0 hthresh=0 wthresh=0 TX queues=1 - TX desc=512 - TX free threshold=0 TX threshold registers: pthresh=0 hthresh=0 wthresh=0 TX RS bit threshold=0 - TXQ flags=0x0 port 1: CRC stripping enabled RX queues=1 - RX desc=128 - RX free threshold=0 RX threshold registers: pthresh=0 hthresh=0 wthresh=0 TX queues=1 - TX desc=512 - TX free threshold=0 TX threshold registers: pthresh=0 hthresh=0 wthresh=0 TX RS bit threshold=0 - TXQ flags=0x0 testpmd> stop Telling cores to stop... Waiting for lcores to finish...

---------------------- Forward statistics for port 0 ---------------------- RX-packets: 16852 RX-dropped: 0 RX-total: 16852 TX-packets: 11469 TX-dropped: 0 TX-total: 11469

---------------------- Forward statistics for port 1 ---------------------- RX-packets: 11473 RX-dropped: 0 RX-total: 11473 TX-packets: 16848 TX-dropped: 0 TX-total: 16848

+++++++++++++++ Accumulated forward statistics for all ports+++++++++++++++ RX-packets: 28325 RX-dropped: 0 RX-total: 28325 TX-packets: 28317 TX-dropped: 0 TX-total: 28317 ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++

Done. emmm, Does the virtual machine environment not support this program？

DubheStar commented 5 years ago

My DPDK NIC has no packet,The cause of this problem is the network card will no packets?

vipinpv85 commented 5 years ago

I have already answered the query with the solution. did you try updating 'configure' for VM nic?

vipinpv85 commented 5 years ago

I have tested on host and Virtual Machine with Intel NIC

DubheStar commented 5 years ago

Sorry. How do I update configure. I don't know how to modify it. :(

vipinpv85 commented 5 years ago

have worked on

  1. suricata before?
  2. autoconf & automake?
  3. DPDK application custom build?
DubheStar commented 5 years ago

I am sorry to make you feel embarrassed. Actually,I need to build a NIDS in 10Gb/s environment .I have less development experience.

And the 82545EM should be a intel NIC.Do I need to add a configuration? if not,please how add non intel NIC in configure

vipinpv85 commented 5 years ago

please share details for ssh to my mail id. let me try to debug online

vipinpv85 commented 5 years ago

you can use https://github.com/vipinpv85/DPDK-Suricata_3.0/issues/11

tolunFdancer commented 5 years ago

@DubheStar Hi, I face the same problem just as you said : 1/8/2019 -- 10:42:40 - - [ERRCODE: SC_ERR_DPDKINTEL_CONFIG_FAILED(275)] - No Mapping found for Port: 1 1/8/2019 -- 10:42:40 - - DPDK Version: DPDK 18.02.2 1/8/2019 -- 10:42:40 - - ----- Global DPDK-INTEL Config ----- 1/8/2019 -- 10:42:40 - - Number Of Ports : 2 1/8/2019 -- 10:42:40 - - Operation Mode : IDS 1/8/2019 -- 10:42:40 - - Port:0, Map:0 1/8/2019 -- 10:42:40 - - Port:1, Map:0 1/8/2019 -- 10:42:40 - - ------------------------------------ 1/8/2019 -- 10:42:40 - - ----- Match Pattern ---- 1/8/2019 -- 10:42:40 - - http: 1 1/8/2019 -- 10:42:40 - - ftp: 0 1/8/2019 -- 10:42:40 - - tls: 0 1/8/2019 -- 10:42:40 - - dns: 0 1/8/2019 -- 10:42:40 - - smtp: 0 1/8/2019 -- 10:42:40 - - ssh: 0 1/8/2019 -- 10:42:40 - - smb: 0 1/8/2019 -- 10:42:40 - - smb2: 0 1/8/2019 -- 10:42:40 - - dcerpc:0 1/8/2019 -- 10:42:40 - - tcp: 1 1/8/2019 -- 10:42:40 - - udp: 0 1/8/2019 -- 10:42:40 - - sctp: 0 1/8/2019 -- 10:42:40 - - icmpv6:0 1/8/2019 -- 10:42:40 - - gre: 0 1/8/2019 -- 10:42:40 - - raw: 0 1/8/2019 -- 10:42:40 - - ipv4: 0 1/8/2019 -- 10:42:40 - - * ipv6: 0 1/8/2019 -- 10:42:40 - - ----------------------- 1/8/2019 -- 10:42:40 - - all 1 packet processing threads, 4 management threads initialized, engine started. 1/8/2019 -- 10:42:40 - - [ERRCODE: SC_ERR_DPDKINTEL_CONFIG_FAILED(275)] - Unknown speed (1000) for 0

I want to konw if you already fixed the issue. If you do ,please tell me something , Thanks!

vipinpv85 commented 4 years ago

Problem definition is not clear. Requested multiple times to cross check the environment. Still waiting to hear back.