search

vipinpv85 / DPDK-Suricata_3.0

add dpdk interface and packet processing to suricata in worker mode
https://github.com/vipinpv85/DPDK-Suricata_3.0
GNU Lesser General Public License v3.0
62 stars 34 forks source link

Error： Ports in DPDK 1 Config-file 2 #11

Closed tolunFdancer closed 5 years ago

tolunFdancer commented 5 years ago

When I run "./src/suricata -c suricata.yaml -s --dpdkintel" it shows the following info: 22/7/2019 -- 15:49:25 - - DPDK Version: DPDK 18.02.2 22/7/2019 -- 15:49:25 - - ----- Global DPDK-INTEL Config ----- 22/7/2019 -- 15:49:25 - - Number Of Ports : 2 22/7/2019 -- 15:49:25 - - Operation Mode : IDS 22/7/2019 -- 15:49:25 - - Port:0, Map:0 22/7/2019 -- 15:49:25 - - Port:0, Map:0 22/7/2019 -- 15:49:25 - - ------------------------------------ 22/7/2019 -- 15:49:25 - - [ERRCODE: SC_ERR_DPDKINTEL_CONFIG_FAILED(275)] - Ports in DPDK 1 Config-file 2 22/7/2019 -- 15:49:25 - - [ERRCODE: SC_ERR_MISSING_CONFIG_PARAM(118)] - DPDK devp!!!

And the suricata.yaml info:

dpdkintel support

   dpdkintel:

   inputs:
      - interface: 0000:09:00.0
        copy-interface: 0000:0a:00.0
      - interface: 0000:0a:00.0
        copy-interface: 0000:09:00.0

    # Select dpdk intel operation mode ips|ids|bypass
        opmode: ids

I do not know why the number of ports is 2, and I do not know where to update it.
And how can I fix this problem ?

tolunFdancer commented 5 years ago

Ubuntu 16.04.6 LTS gcc version 5.4.0 dpdk 18.02.2

tolunFdancer commented 5 years ago

./src/suricata --list-dpdkintel-ports: EAL: Detected 4 lcore(s) EAL: Multi-process socket /var/run/.rte_unix EAL: Probing VFIO support... EAL: PCI device 0000:00:19.0 on NUMA socket -1 EAL: Invalid NUMA socket, default to 0 EAL: probe driver: 8086:15a3 net_e1000_em

--- DPDK Intel Ports ---

vipinpv85 commented 5 years ago

you have only 1 interface in DPDK, hence your yaml entry should be

#dpdkintel support
dpdkintel:

   inputs:
      - interface: 0000:09:00.0
        copy-interface: 0000:09:00.0

 # Select dpdk intel operation mode ips|ids|bypass
        opmode: ids
tolunFdancer commented 5 years ago

Then comes the following info: alled to add eve-log support. 23/7/2019 -- 10:57:27 - - DPDK Version: DPDK 18.02.2 23/7/2019 -- 10:57:27 - - ----- Global DPDK-INTEL Config ----- 23/7/2019 -- 10:57:27 - - Number Of Ports : 1 23/7/2019 -- 10:57:27 - - Operation Mode : IDS 23/7/2019 -- 10:57:27 - - Port:0, Map:0 23/7/2019 -- 10:57:27 - - ------------------------------------ 23/7/2019 -- 10:57:27 - - [ERRCODE: SC_ERR_MISSING_CONFIG_PARAM(118)] - port:0; duplex:half, status:up 23/7/2019 -- 10:57:27 - - [ERRCODE: SC_ERR_MISSING_CONFIG_PARAM(118)] - DPDK dev setup!!!

vipinpv85 commented 5 years ago

run with ./src/suricata -c suricata.yaml -s <myrules.rules> --dpdkintel -vvv and share details

vipinpv85 commented 5 years ago

I have a check link.link_duplex != ETH_LINK_FULL_DUPLEX. currently, your interface is half-duplex hence it is failing. Try to fix this to full-duplex or so for time being for your test you can comment the return value in file: src/util-dpdk-setup.c function: dpdkIntelDevSetup line: 219 rebuild and try

tolunFdancer commented 5 years ago

This time I build dpdk_suricata3.0 in centos，because finally I will run my application in centos. When I run "./src/suricata --list-dpdkintel-ports" , shows the following info: EAL: Detected 8 lcore(s) EAL: Multi-process socket /var/run/.rte_unix EAL: Probing VFIO support... EAL: VFIO support initialized

--- DPDK Intel Ports ---

I wanna know what is dpdk intel ports , why the overal ports is 0 and how to configure it.

The details.txt is the info when I run "./src/suricata -c suricata.yaml -s /data/tolun/DPDK-Suricata_3.0-master/suricata-3.0/rules/test-baidu.rules --dpdkintel" details.txt

vipinpv85 commented 5 years ago

if the result of './src/suricata --list-dpdkintel-ports' is '0'. It means there are not DPDK ports bound with 'igb_uio, uio_pci_generic or vfio_pci'.

  1. Use dpdk tool ./usertools/dpdk-devbind.py -s to find if there are any binded devices?
  2. If there are bound device, check if your suricata.yaml pcie-address matches with bound devices. If no correct the entries.

errors:

23/7/2019 -- 18:18:36 - <Notice> - DPDK Version: DPDK 18.02.2
23/7/2019 -- 18:18:36 - <Notice> - ----- Global DPDK-INTEL Config -----
23/7/2019 -- 18:18:36 - <Notice> -  Number Of Ports  : 0
23/7/2019 -- 18:18:36 - <Notice> -  Operation Mode   : IPS
23/7/2019 -- 18:18:36 - <Notice> - ------------------------------------
23/7/2019 -- 18:18:36 - <Error> - [ERRCODE: SC_ERR_DPDKINTEL_CONFIG_FAILED(275)] -  Ports in DPDK 0 Config-file 0
  1. dpdk version.
  2. number of ports
tolunFdancer commented 5 years ago

Thanks! The dpdk binds device success, here is the info when I run "./usertools/dpdk-devbind.py -s": Network devices using DPDK-compatible driver 0000:0b:00.0 'VMXNET3 Ethernet Controller 07b0' drv=igb_uio unused=vmxnet3,vfio-pci,uio_pci_generic

Network devices using kernel driver 0000:03:00.0 'VMXNET3 Ethernet Controller 07b0' if=ens160 drv=vmxnet3 unused=igb_uio,vfio-pci,uio_pci_generic Active

And the suricata.yaml info is : dpdkintel: inputs:

If my configure is wrong somewhere? Still comes the error, see in the details.txt. details.txt

vipinpv85 commented 5 years ago

Dpdk is binded for vmxnet3 and not Intel nic or virtio. The configure and Akefile is build for Intel nic and virtio.

If you want vmxnet3 support please edit configure.ac and Makefile.am appropriately.

tolunFdancer commented 5 years ago

In configure.ac, line 1154 and 1165 exists "-Wl,-lrte_pmd_virtio" , should I change "-Wl,-lrte_pmd_virtio" to "-Wl,-lrte_pmd_vmxnet3"？ And in makefile.am , I did not find something about nic virtio and vmxnet3. So can you teach me how to edit configure.ac and Makefile.am appropriately? Thank you very much!

vipinpv85 commented 5 years ago

Instead of removing you should be adding -Wl,-lrte_pmd_vmxnet3.

tolunFdancer commented 5 years ago

I edited the configure.ac as you tole me, Then I run " autoconf, ./configure --enable-dpdkintel, make -j 10", but when I run "make -j 10 " it turns out the following error: ./configure: line 12099: syntax error near unexpected token 0.21' ./configure: line 12099: PKG_PROG_PKG_CONFIG(0.21) # 0.21 is the CentOS 5.11 version' make: *** [config.status] Error 2

centos version is : CentOS Linux release 7.3.1611 (Core) And I wanna konw if the makefile.am needs edit?

vipinpv85 commented 5 years ago

@tolunFdancer please check your steps. as per the error log ./configure: line 12099: syntax error near unexpected token 0.21' ./configure: line 12099: you have received the error while running ./configure --enable-dpdkintel and not make -j 10

send me diff of your change

tolunFdancer commented 5 years ago

Yeah, the error happens in " ./configure --enable-dpdkintel" . In configure.ac line 1154, I made the chane "-lrte_pmd_virtio -Wl,-lrte_pmd_vmxnet3 -Wl," and line 1165 "-lrte_pmd_virtio -Wl,-lrte_pmd_vmxnet3 -Wl,".

vipinpv85 commented 5 years ago

@tolunFdancer use diff by git git diff configure.ac.

did you try to check what is error in line 12099?

quick question, did you build DPDK for vmxnet3? if yes, try adding -lrte_pmd_vmxnet3_uio -Wl instead of -lrte_pmd_vmxnet3 -Wl

tolunFdancer commented 5 years ago

I build DPDK with following steps: export RTE_SDK=/root/dpdk-stable-17.08.1 export RTE_TARGET=x86_64-native-linuxapp-gcc make config T=x86_64-native-linuxapp-gcc make make install T=x86_64-native-linuxapp-gcc modprobe uio_pci_generic modprobe uio modprobe vfio-pci insmod build/kmod/igb_uio.ko

Then I bind the device with dpdk. I did not konw if I build DPDK for vmxnet3, I am not familiar with DPDK. And the error in 12099, I do not know the meaning of it.

vipinpv85 commented 5 years ago

the line 12099 is from configure script of suricata and not DPDK. I think the issue is from what you have added in configure.ac. hence solution is try adding -lrte_pmd_vmxnet3_uio -Wl instead of -lrte_pmd_vmxnet3 -Wl

vipinpv85 commented 5 years ago

please let me know your update

tolunFdancer commented 5 years ago

Still do not work. checking whether to build shared libraries... yes checking whether to build static libraries... yes ./configure: line 12099: syntax error near unexpected token 0.21' ./configure: line 12099: PKG_PROG_PKG_CONFIG(0.21) # 0.21 is the CentOS 5.11 version'

vipinpv85 commented 5 years ago

I do not think this is DPDK flag add issue, it looks like your new environment is missing some packages. Easiest way to dissect this is

  1. Run without --enable-dpdkintel
  2. If failure is seen, then check config.log which package the enviroment is failed for.
  3. Install the package, retry with --enable-dpdkintel
vipinpv85 commented 5 years ago

if possible share your config.log.

tolunFdancer commented 5 years ago

Run "./configure" success, and run "./configure --enable-dpdkintel" success, when I run "make" the error shows, more info in details.txt. At the begining of details.txt, it shows "missing aclocal-1.15", I already download automake-1.15 and build it . details.txt config.log

vipinpv85 commented 5 years ago

can you try quickly by running autoreconf -f -i. then repeat the steps

reference https://stackoverflow.com/questions/33278928/how-to-overcome-aclocal-1-15-is-missing-on-your-system-warning

vipinpv85 commented 5 years ago

So the steps are

  1. autoreconf -f -i
  2. autoconf
  3. ./configure --enable-dpdkintel
vipinpv85 commented 5 years ago

I have added -lrte_pmd_vmxnet3_uio -Wl in my local branch. result is 


Suricata Configuration:
  AF_PACKET support:                       no
  DPDK_INTEL support:                      yes
  PF_RING support:                         no
  NFQueue support:                         no
  NFLOG support:                           no
  IPFW support:                            no
  Netmap support:                          no
  DAG enabled:                             no
  Napatech enabled:                        no

make result

mkdir -p ../../scripts/suricatasc/src
./setup.py build;
running build
running build_py
creating build
creating build/lib.linux-x86_64-2.7
creating build/lib.linux-x86_64-2.7/suricatasc
copying src/suricatasc.py -> build/lib.linux-x86_64-2.7/suricatasc
copying src/__init__.py -> build/lib.linux-x86_64-2.7/suricatasc
running build_scripts
creating build/scripts-2.7
copying and adjusting suricatasc -> build/scripts-2.7
changing mode of build/scripts-2.7/suricatasc from 644 to 755
make[3]: Leaving directory '/suricata/DPDK-Suricata_3.0/suricata-3.0/scripts/suricatasc'
make[3]: Entering directory '/suricata/DPDK-Suricata_3.0/suricata-3.0/scripts'
make[3]: Nothing to be done for 'all-am'.
make[3]: Leaving directory '/suricata/DPDK-Suricata_3.0/suricata-3.0/scripts'
make[2]: Leaving directory '/suricata/DPDK-Suricata_3.0/suricata-3.0/scripts'
make[2]: Entering directory '/suricata/DPDK-Suricata_3.0/suricata-3.0'
make[2]: Leaving directory '/suricata/DPDK-Suricata_3.0/suricata-3.0'
make[1]: Leaving directory '/suricata/DPDK-Suricata_3.0/suricata-3.0'

So it is environment-dependent, there is nothing wrong with DPDK addition or configure.ac

vipinpv85 commented 5 years ago

@tolunFdancer there is suricata reported bug on similar issue with centos 5 (End of Life) https://github.com/OISF/suricata/pull/2679 https://redmine.openinfosecfoundation.org/issues/1759

you should use OS which is supported or apply the workaround shared in URL

CentOS 5 will be EOL on 31.3.2017: "Maintenance Updates Q1, 2011 - Mar 31st, 2017" (https://wiki.centos.org/FAQ/CentOS5)

Thing that can be removed afterwards:

vipinpv85 commented 5 years ago

please do the needful there is nothing more from my end to support on

tolunFdancer commented 5 years ago

But my centos version is : CentOS Linux release 7.3.1611. When I run "autoreconf -f -i", it comes the following info: sh: ./get-version.sh: Permission denied configure.ac:7: error: AC_INIT should be called with package and version arguments /usr/local/share/aclocal-1.15/init.m4:29: AM_INIT_AUTOMAKE is expanded from... configure.ac:7: the top level autom4te: /usr/bin/m4 failed with exit status: 1 aclocal: error: echo failed with exit status: 1 autoreconf: aclocal failed with exit status: 1

vipinpv85 commented 5 years ago

from details.log

./configure: line 12099: syntax error near unexpected token 0.21' ./configure: line 12099: PKG_PROG_PKG_CONFIG(0.21) # 0.21 is the CentOS 5.11 version'

once again it works without issue for configure, make, execute. this is related to your environment.

share your ip address and username:password in email.

tolunFdancer commented 5 years ago

Thanks! I sorry I can not share my address and username:password with you because the job situation, I hope you understand it .

vipinpv85 commented 5 years ago

thanks for the reply. No, I do not understand.

There is nothing more I can help on this ticket. Since the error is from your environment setup

https://lists.openinfosecfoundation.org/pipermail/oisf-users/2016-March/013770.html

tolunFdancer commented 5 years ago

I know you wanna help me to find out the enviroment problem, but my company did not allow me share the ip address and username:pwd info from somebody outside.

vipinpv85 commented 5 years ago

looks liek your pkg-config is faulty, you refer to the url

tolunFdancer commented 5 years ago

Thanks for your advice , truelly! I fixed the pkg-config problem. Now my DPDK Intel Ports is one ,here is the info when I run "./src/suricata --list-dpdkintel-ports": --- DPDK Intel Ports ---

When I run "./src/suricata -c suricata.yaml -s /opt/tolun/DPDK-Suricata_3.0-master/suricata-3.0/rules/test-baidu.rules --dpdkintel", it turns out "port:0; duplex:half, status:down".

I comment the return value in file: src/util-dpdk-setup.c function: dpdkIntelDevSetup line: 219 but still do not work , more info in details.txt. So do I have to make half-duplex to full-duplex? details.txt

vipinpv85 commented 5 years ago

cool, Please open a new ticket for the same

quick reply: 25/7/2019 -- 16:10:20 - - [ERRCODE: SC_ERR_DPDKINTEL_CONFIG_FAILED(275)] - Unknown speed (0) for 0