search

vipinpv85 / DPDK-Suricata_3.0

add dpdk interface and packet processing to suricata in worker mode
https://github.com/vipinpv85/DPDK-Suricata_3.0
GNU Lesser General Public License v3.0
64 stars 34 forks source link

I compiled an error using dpdk-16.07 #4

Closed duanrui123456 closed 5 years ago

duanrui123456 commented 5 years ago

1、tar xf dpdk-16.07.tar.xz cd dpdk-16.07 make install T=x86_64-native-linuxapp-gcc

To set the environment variables
export RTE_SDK=/home/dpdk-16.07 export RTE_TARGET=x86_64-native-linuxapp-gcc

Then to do the following test DPDK operation, compile and run normally cd /home/dpdk-16.07/examples/helloworld make cd build ./helloworld

2、 unzip DPDK-Suricata_3.0-master.zip cd DPDK-Suricata_3.0-master/suricata-3.0/ ./configure -enable-dpdkintel --with-libdpdkintel-includes=$RTE_SDK/$RTE_TARGET/include/ --with-libdpdkintel-libraries=$RTE_SDK/$RTE_TARGET/lib make -j all then In file included from /home/dpdk-16.07/x86_64-native-linuxapp-gcc/include/rte_mbuf.h:58:0, from /home/dpdk-16.07/x86_64-native-linuxapp-gcc/include/rte_ether.h:52, from dpdk-include-common.h:17, from suricata.c:202: /home/dpdk-16.07/x86_64-native-linuxapp-gcc/include/rte_mempool.h:155:2: error: expected specifier-qualifier-list before ‘STAILQ_ENTRY’ STAILQ_ENTRY(rte_mempool_objhdr) next; /< Next in list. / ^ /home/dpdk-16.07/x86_64-native-linuxapp-gcc/include/rte_mempool.h:166:1: warning: data definition has no type or storage class STAILQ_HEAD(rte_mempool_objhdr_list, rte_mempool_objhdr); ^ /home/dpdk-16.07/x86_64-native-linuxapp-gcc/include/rte_mempool.h:166:1: warning: type defaults to ‘int’ in declaration of ‘STAILQ_HEAD’ [-Wimplicit-int] /home/dpdk-16.07/x86_64-native-linuxapp-gcc/include/rte_mempool.h:166:1: warning: parameter names (without types) in function declaration /home/dpdk-16.07/x86_64-native-linuxapp-gcc/include/rte_mempool.h:185:1: warning: data definition has no type or storage class STAILQ_HEAD(rte_mempool_memhdr_list, rte_mempool_memhdr); ^ /home/dpdk-16.07/x86_64-native-linuxapp-gcc/include/rte_mempool.h:185:1: warning: type defaults to ‘int’ in declaration of ‘STAILQ_HEAD’ [-Wimplicit-int] /home/dpdk-16.07/x86_64-native-linuxapp-gcc/include/rte_mempool.h:185:1: warning: parameter names (without types) in function declaration /home/dpdk-16.07/x86_64-native-linuxapp-gcc/include/rte_mempool.h:191:2: warning: ‘struct rte_mempool_memhdr’ declared inside parameter list void opaque); ^ /home/dpdk-16.07/x86_64-native-linuxapp-gcc/include/rte_mempool.h:191:2: warning: its scope is only this definition or declaration, which is probably not what you want /home/dpdk-16.07/x86_64-native-linuxapp-gcc/include/rte_mempool.h:200:2: error: expected specifier-qualifier-list before ‘STAILQ_ENTRY’ STAILQ_ENTRY(rte_mempool_memhdr) next; /< Next in list. */ ^ /home/dpdk-16.07/x86_64-native-linuxapp-gcc/include/rte_mempool.h:248:33: error: field ‘elt_list’ has incomplete type struct rte_mempool_objhdr_list elt_list; /< List of objects in pool */ ^ /home/dpdk-16.07/x86_64-native-linuxapp-gcc/include/rte_mempool.h:250:33: error: field ‘mem_list’ has incomplete type struct rte_mempool_memhdr_list mem_list; /< List of memory chunks / ^ /home/dpdk-16.07/x86_64-native-linuxapp-gcc/include/rte_mempool.h: In function ‘rte_mempool_from_obj’: /home/dpdk-16.07/x86_64-native-linuxapp-gcc/include/rte_mempool.h:318:12: error: ‘struct rte_mempool_objhdr’ has no member named ‘mp’ return hdr->mp; ^ /home/dpdk-16.07/x86_64-native-linuxapp-gcc/include/rte_mempool.h: In function ‘rte_mempool_virt2phy’: /home/dpdk-16.07/x86_64-native-linuxapp-gcc/include/rte_mempool.h:1624:12: error: ‘const struct rte_mempool_objhdr’ has no member named ‘physaddr’ return hdr->physaddr; ^ Makefile:1639: recipe for target 'suricata.o' failed make[3]: [suricata.o] Error 1 make[3]: Leaving directory '/home/DPDK-Suricata_3.0-master/suricata-3.0/src' Makefile:1139: recipe for target 'all' failed make[2]: * [all] Error 2 make[2]: Leaving directory '/home/DPDK-Suricata_3.0-master/suricata-3.0/src' Makefile:468: recipe for target 'all-recursive' failed make[1]: [all-recursive] Error 1 make[1]: Leaving directory '/home/DPDK-Suricata_3.0-master/suricata-3.0' Makefile:398: recipe for target 'all' failed make: [all] Error 2

This is the environment in which I operate

vipinpv85 commented 5 years ago

Hi,

I humbly request to use issue template while opening a ticket.

Based on the debug logs shared, we can see

**/home/dpdk-16.07/x86_64-native-linuxapp-gcc/include/rte_mempool.h:200:2: error: expected specifier-qualifier-list before ‘STAILQ_ENTRY’ STAILQ_ENTRY(rte_mempool_memhdr) next; /**< Next in list. */
^
/home/dpdk-16.07/x86_64-native-linuxapp-gcc/include/rte_mempool.h:248:33: error: field ‘elt_list’ has incomplete type
struct rte_mempool_objhdr_list elt_list; /**< List of objects in pool */ ^ /home/dpdk-16.07/x86_64-native-linuxapp-gcc/include/rte_mempool.h:250:33: error: field ‘mem_list’ has incomplete type struct rte_mempool_memhdr_list mem_list; /**< List of memory chunks */
^
/home/dpdk-16.07/x86_64-native-linuxapp-gcc/include/rte_mempool.h: In function ‘rte_mempool_from_obj’:
/home/dpdk-16.07/x86_64-native-linuxapp-gcc/include/rte_mempool.h:318:12: error: ‘struct rte_mempool_objhdr’ has no member named ‘mp’
return hdr->mp;
^
/home/dpdk-16.07/x86_64-native-linuxapp-gcc/include/rte_mempool.h: In function ‘rte_mempool_virt2phy’:
/home/dpdk-16.07/x86_64-native-linuxapp-gcc/include/rte_mempool.h:1624:12: error: ‘const struct rte_mempool_objhdr’ has no member named ‘physaddr’
return hdr->physaddr;

These look like DPDk build issues, can you cross check if DPDK and Examples are built fine?

note: I will keep this open, please confirm the same

duanrui123456 commented 5 years ago

Thank you for your reply:

I have tested the example, which can be used by normal compilation. I humbly request your help. Could you please write down your test process in detail to help us quickly apply your program?

vipinpv85 commented 5 years ago

Hi,

once again i request you share the details in template format (Issue.template) with configure logs

vipinpv85 commented 5 years ago

Hi,

DPDK: 16.07.2 uname: 4.15.0-45-generic

the issue in the DPDK build: 

/home/saesrv02/Downloads/explore/suricata/dpdk-stable-16.07.2/x86_64-native-linuxapp-gcc/build/lib/librte_eal/linuxapp/igb_uio/igb_uio.c:383:7: error: implicit declaration of function ‘pci_enable_msix’; did you mean ‘pci_enable_msi’? [-Werror=implicit-function-declaration]
   if (pci_enable_msix(dev, &msix_entry, 1) == 0) {
       ^~~~~~~~~~~~~~~
       pci_enable_msi

So unless you update and share the logs as per template it is difficult for me to follow your issue.

Please update the issue in the template with configure.log. I will keep this open for one more day

vipinpv85 commented 5 years ago

thanks for editing, can please share the Linux version or terminal access to your box.

Once again the error is from DPDK In file included from /home/dpdk-16.07/x86_64-native-linuxapp-gcc/include/rte_mbuf.h:58:0, from /home/dpdk-16.07/x86_64-native-linuxapp-gcc/include/rte_ether.h:52, from dpdk-include-common.h:17, from suricata.c:202: /home/dpdk-16.07/x86_64-native-linuxapp-gcc/include/rte_mempool.h:155:2: error: expected specifier-qualifier-list before ‘STAILQ_ENTRY’ STAILQ_ENTRY(rte_mempool_objhdr) next; /**< Next in list. */**

could this be due to GCC on how it compiles?

vipinpv85 commented 5 years ago

my suggestion is try adding keyword 'struct' in '/home/dpdk-16.07/x86_64-native-linuxapp-gcc/include/rte_mempool.h:155'

result would be 

STAILQ_ENTRY(struct rte_mempool_memhdr) next

since error states error: expected specifier-qualifier-list before ‘STAILQ_ENTRY’. Once again I think it from the compiler, what is the compiler version in use?

or try replacing STAILQ_ENTRY to TAILQ_ENTRY

vipinpv85 commented 5 years ago

One suggestion 

cd build
./helloworld

Hello-world application does not make use of mbuf, mempool or other libraries.

duanrui123456 commented 5 years ago

Thank you for your reply

  1. This is the environment in which I operate 
    * OS：ubuntu 16.04.2
* GCC：version 5.4.0
* DPDK Version [ 16.07]
- uname：4.4.0-62-generic
  2. L2fwd has been tested and can be used normally 
    ./l2fwd -c 3 -n 2 -- -p 1 -T 1

    3.I have carried out the test according to your advice, but I still get an error

4.I compiled the DPDK directly from the following command

tar xf dpdk-16.07.tar.xz
cd dpdk-16.07
make install T=x86_64-native-linuxapp-gcc

5.Can you list your operating environment and give detailed installation steps?

vipinpv85 commented 5 years ago

5.Can you list your operating environment and give detailed installation steps?

README

make install T=x86_64-native-linuxapp-gcc

make install T=x86_64-native-linuxapp-gcc O=x86_64-native-linuxapp-gcc.

Then update RTE_SDK and RTE_TARGET, cd $RTE_SDK/$RTE_TARGET/, make, cd $RTE_SDK/l2fwd, make.

Option 1: update DPDK to 17 series Option 2: if you search for qualifier error there are enough examples explaining why so.

Note: I still have not ssh access to your terminal to understand your issue yet

vipinpv85 commented 5 years ago

Can you output 

ls $RTE_SDK/$RTE_TARGET/include/
ls $RTE_SDK/$RTE_TARGET/lib/
duanrui123456 commented 5 years ago

this is dpdk-17.08.2 include：

drwxr-xr-x 4 root root  4096 Feb 20 22:24 ./
drwxr-xr-x 7 root root  4096 Feb 20 22:22 ../
lrwxrwxrwx 1 root root    44 Feb 20 22:22 cmdline_cirbuf.h -> ../../../lib/librte_cmdline/cmdline_cirbuf.h
lrwxrwxrwx 1 root root    37 Feb 20 22:22 cmdline.h -> ../../../lib/librte_cmdline/cmdline.h
lrwxrwxrwx 1 root root    53 Feb 20 22:22 cmdline_parse_etheraddr.h -> ../../../lib/librte_cmdline/cmdline_parse_etheraddr.h
lrwxrwxrwx 1 root root    43 Feb 20 22:22 cmdline_parse.h -> ../../../lib/librte_cmdline/cmdline_parse.h
lrwxrwxrwx 1 root root    50 Feb 20 22:22 cmdline_parse_ipaddr.h -> ../../../lib/librte_cmdline/cmdline_parse_ipaddr.h
lrwxrwxrwx 1 root root    47 Feb 20 22:22 cmdline_parse_num.h -> ../../../lib/librte_cmdline/cmdline_parse_num.h
lrwxrwxrwx 1 root root    52 Feb 20 22:22 cmdline_parse_portlist.h -> ../../../lib/librte_cmdline/cmdline_parse_portlist.h
lrwxrwxrwx 1 root root    50 Feb 20 22:22 cmdline_parse_string.h -> ../../../lib/librte_cmdline/cmdline_parse_string.h
lrwxrwxrwx 1 root root    44 Feb 20 22:22 cmdline_rdline.h -> ../../../lib/librte_cmdline/cmdline_rdline.h
lrwxrwxrwx 1 root root    44 Feb 20 22:22 cmdline_socket.h -> ../../../lib/librte_cmdline/cmdline_socket.h
lrwxrwxrwx 1 root root    43 Feb 20 22:22 cmdline_vt100.h -> ../../../lib/librte_cmdline/cmdline_vt100.h
drwxr-xr-x 2 root root  4096 Feb 20 22:22 exec-env/
drwxr-xr-x 2 root root  4096 Feb 20 22:22 generic/
lrwxrwxrwx 1 root root    33 Feb 20 22:22 rte_acl.h -> ../../../lib/librte_acl/rte_acl.h
lrwxrwxrwx 1 root root    39 Feb 20 22:22 rte_acl_osdep.h -> ../../../lib/librte_acl/rte_acl_osdep.h
lrwxrwxrwx 1 root root    50 Feb 20 22:22 rte_alarm.h -> ../../../lib/librte_eal/common/include/rte_alarm.h
lrwxrwxrwx 1 root root    38 Feb 20 22:22 rte_approx.h -> ../../../lib/librte_sched/rte_approx.h
lrwxrwxrwx 1 root root    33 Feb 20 22:22 rte_arp.h -> ../../../lib/librte_net/rte_arp.h
lrwxrwxrwx 1 root root    63 Feb 20 22:22 rte_atomic_32.h -> ../../../lib/librte_eal/common/include/arch/x86/rte_atomic_32.h
lrwxrwxrwx 1 root root    63 Feb 20 22:22 rte_atomic_64.h -> ../../../lib/librte_eal/common/include/arch/x86/rte_atomic_64.h
lrwxrwxrwx 1 root root    60 Feb 20 22:22 rte_atomic.h -> ../../../lib/librte_eal/common/include/arch/x86/rte_atomic.h
lrwxrwxrwx 1 root root    41 Feb 20 22:23 rte_avp_common.h -> ../../../drivers/net/avp/rte_avp_common.h
lrwxrwxrwx 1 root root    39 Feb 20 22:23 rte_avp_fifo.h -> ../../../drivers/net/avp/rte_avp_fifo.h
lrwxrwxrwx 1 root root    38 Feb 20 22:22 rte_bitmap.h -> ../../../lib/librte_sched/rte_bitmap.h
lrwxrwxrwx 1 root root    46 Feb 20 22:22 rte_bitrate.h -> ../../../lib/librte_bitratestats/rte_bitrate.h
lrwxrwxrwx 1 root root    62 Feb 20 22:22 rte_branch_prediction.h -> ../../../lib/librte_eal/common/include/rte_branch_prediction.h
lrwxrwxrwx 1 root root    48 Feb 20 22:22 rte_bus.h -> ../../../lib/librte_eal/common/include/rte_bus.h
lrwxrwxrwx 1 root root    66 Feb 20 22:22 rte_byteorder_32.h -> ../../../lib/librte_eal/common/include/arch/x86/rte_byteorder_32.h
lrwxrwxrwx 1 root root    66 Feb 20 22:22 rte_byteorder_64.h -> ../../../lib/librte_eal/common/include/arch/x86/rte_byteorder_64.h
lrwxrwxrwx 1 root root    63 Feb 20 22:22 rte_byteorder.h -> ../../../lib/librte_eal/common/include/arch/x86/rte_byteorder.h
lrwxrwxrwx 1 root root    41 Feb 20 22:22 rte_cfgfile.h -> ../../../lib/librte_cfgfile/rte_cfgfile.h
lrwxrwxrwx 1 root root    51 Feb 20 22:22 rte_common.h -> ../../../lib/librte_eal/common/include/rte_common.h
lrwxrwxrwx 1 root root    39 Feb 20 22:22 rte_compat.h -> ../../../lib/librte_compat/rte_compat.h
-rw-r--r-- 1 root root 13807 Feb 20 22:22 rte_config.h
lrwxrwxrwx 1 root root    62 Feb 20 22:22 rte_cpuflags.h -> ../../../lib/librte_eal/common/include/arch/x86/rte_cpuflags.h
lrwxrwxrwx 1 root root    45 Feb 20 22:22 rte_cryptodev.h -> ../../../lib/librte_cryptodev/rte_cryptodev.h
lrwxrwxrwx 1 root root    49 Feb 20 22:22 rte_cryptodev_pci.h -> ../../../lib/librte_cryptodev/rte_cryptodev_pci.h
lrwxrwxrwx 1 root root    49 Feb 20 22:22 rte_cryptodev_pmd.h -> ../../../lib/librte_cryptodev/rte_cryptodev_pmd.h
lrwxrwxrwx 1 root root    59 Feb 20 22:24 rte_cryptodev_scheduler.h -> ../../../drivers/crypto/scheduler/rte_cryptodev_scheduler.h
lrwxrwxrwx 1 root root    70 Feb 20 22:24 rte_cryptodev_scheduler_operations.h -> ../../../drivers/crypto/scheduler/rte_cryptodev_scheduler_operations.h
lrwxrwxrwx 1 root root    50 Feb 20 22:22 rte_cryptodev_vdev.h -> ../../../lib/librte_cryptodev/rte_cryptodev_vdev.h
lrwxrwxrwx 1 root root    42 Feb 20 22:22 rte_crypto.h -> ../../../lib/librte_cryptodev/rte_crypto.h
lrwxrwxrwx 1 root root    46 Feb 20 22:22 rte_crypto_sym.h -> ../../../lib/librte_cryptodev/rte_crypto_sym.h
lrwxrwxrwx 1 root root    60 Feb 20 22:22 rte_cycles.h -> ../../../lib/librte_eal/common/include/arch/x86/rte_cycles.h
lrwxrwxrwx 1 root root    50 Feb 20 22:22 rte_debug.h -> ../../../lib/librte_eal/common/include/rte_debug.h
lrwxrwxrwx 1 root root    52 Feb 20 22:22 rte_devargs.h -> ../../../lib/librte_eal/common/include/rte_devargs.h
lrwxrwxrwx 1 root root    48 Feb 20 22:22 rte_dev.h -> ../../../lib/librte_eal/common/include/rte_dev.h
lrwxrwxrwx 1 root root    40 Feb 20 22:22 rte_dev_info.h -> ../../../lib/librte_ether/rte_dev_info.h
lrwxrwxrwx 1 root root    49 Feb 20 22:22 rte_distributor.h -> ../../../lib/librte_distributor/rte_distributor.h
lrwxrwxrwx 1 root root    48 Feb 20 22:22 rte_eal.h -> ../../../lib/librte_eal/common/include/rte_eal.h
lrwxrwxrwx 1 root root    58 Feb 20 22:22 rte_eal_memconfig.h -> ../../../lib/librte_eal/common/include/rte_eal_memconfig.h
lrwxrwxrwx 1 root root    33 Feb 20 22:22 rte_efd.h -> ../../../lib/librte_efd/rte_efd.h
lrwxrwxrwx 1 root root    50 Feb 20 22:22 rte_errno.h -> ../../../lib/librte_eal/common/include/rte_errno.h
lrwxrwxrwx 1 root root    50 Feb 20 22:23 rte_eth_bond_8023ad.h -> ../../../drivers/net/bonding/rte_eth_bond_8023ad.h
lrwxrwxrwx 1 root root    43 Feb 20 22:23 rte_eth_bond.h -> ../../../drivers/net/bonding/rte_eth_bond.h
lrwxrwxrwx 1 root root    40 Feb 20 22:22 rte_eth_ctrl.h -> ../../../lib/librte_ether/rte_eth_ctrl.h
lrwxrwxrwx 1 root root    38 Feb 20 22:22 rte_ethdev.h -> ../../../lib/librte_ether/rte_ethdev.h
lrwxrwxrwx 1 root root    42 Feb 20 22:22 rte_ethdev_pci.h -> ../../../lib/librte_ether/rte_ethdev_pci.h
lrwxrwxrwx 1 root root    43 Feb 20 22:22 rte_ethdev_vdev.h -> ../../../lib/librte_ether/rte_ethdev_vdev.h
lrwxrwxrwx 1 root root    35 Feb 20 22:22 rte_ether.h -> ../../../lib/librte_net/rte_ether.h
lrwxrwxrwx 1 root root    40 Feb 20 22:24 rte_eth_ring.h -> ../../../drivers/net/ring/rte_eth_ring.h
lrwxrwxrwx 1 root root    42 Feb 20 22:24 rte_eth_vhost.h -> ../../../drivers/net/vhost/rte_eth_vhost.h
lrwxrwxrwx 1 root root    43 Feb 20 22:22 rte_eventdev.h -> ../../../lib/librte_eventdev/rte_eventdev.h
lrwxrwxrwx 1 root root    47 Feb 20 22:22 rte_eventdev_pmd.h -> ../../../lib/librte_eventdev/rte_eventdev_pmd.h
lrwxrwxrwx 1 root root    51 Feb 20 22:22 rte_eventdev_pmd_pci.h -> ../../../lib/librte_eventdev/rte_eventdev_pmd_pci.h
lrwxrwxrwx 1 root root    52 Feb 20 22:22 rte_eventdev_pmd_vdev.h -> ../../../lib/librte_eventdev/rte_eventdev_pmd_vdev.h
lrwxrwxrwx 1 root root    45 Feb 20 22:22 rte_event_ring.h -> ../../../lib/librte_eventdev/rte_event_ring.h
lrwxrwxrwx 1 root root    39 Feb 20 22:22 rte_fbk_hash.h -> ../../../lib/librte_hash/rte_fbk_hash.h
lrwxrwxrwx 1 root root    43 Feb 20 22:22 rte_flow_driver.h -> ../../../lib/librte_ether/rte_flow_driver.h
lrwxrwxrwx 1 root root    36 Feb 20 22:22 rte_flow.h -> ../../../lib/librte_ether/rte_flow.h
lrwxrwxrwx 1 root root    33 Feb 20 22:22 rte_gre.h -> ../../../lib/librte_net/rte_gre.h
lrwxrwxrwx 1 root root    33 Feb 20 22:22 rte_gro.h -> ../../../lib/librte_gro/rte_gro.h
lrwxrwxrwx 1 root root    39 Feb 20 22:22 rte_hash_crc.h -> ../../../lib/librte_hash/rte_hash_crc.h
lrwxrwxrwx 1 root root    35 Feb 20 22:22 rte_hash.h -> ../../../lib/librte_hash/rte_hash.h
lrwxrwxrwx 1 root root    52 Feb 20 22:22 rte_hexdump.h -> ../../../lib/librte_eal/common/include/rte_hexdump.h
lrwxrwxrwx 1 root root    34 Feb 20 22:22 rte_icmp.h -> ../../../lib/librte_net/rte_icmp.h
lrwxrwxrwx 1 root root    55 Feb 20 22:22 rte_interrupts.h -> ../../../lib/librte_eal/common/include/rte_interrupts.h
lrwxrwxrwx 1 root root    56 Feb 20 22:22 rte_io.h -> ../../../lib/librte_eal/common/include/arch/x86/rte_io.h
lrwxrwxrwx 1 root root    41 Feb 20 22:22 rte_ip_frag.h -> ../../../lib/librte_ip_frag/rte_ip_frag.h
lrwxrwxrwx 1 root root    32 Feb 20 22:22 rte_ip.h -> ../../../lib/librte_net/rte_ip.h
lrwxrwxrwx 1 root root    36 Feb 20 22:22 rte_jhash.h -> ../../../lib/librte_hash/rte_jhash.h
lrwxrwxrwx 1 root root    43 Feb 20 22:22 rte_jobstats.h -> ../../../lib/librte_jobstats/rte_jobstats.h
lrwxrwxrwx 1 root root    54 Feb 20 22:22 rte_keepalive.h -> ../../../lib/librte_eal/common/include/rte_keepalive.h
lrwxrwxrwx 1 root root    33 Feb 20 22:22 rte_kni.h -> ../../../lib/librte_kni/rte_kni.h
lrwxrwxrwx 1 root root    39 Feb 20 22:22 rte_kvargs.h -> ../../../lib/librte_kvargs/rte_kvargs.h
lrwxrwxrwx 1 root root    51 Feb 20 22:22 rte_latencystats.h -> ../../../lib/librte_latencystats/rte_latencystats.h
lrwxrwxrwx 1 root root    51 Feb 20 22:22 rte_launch.h -> ../../../lib/librte_eal/common/include/rte_launch.h
lrwxrwxrwx 1 root root    50 Feb 20 22:22 rte_lcore.h -> ../../../lib/librte_eal/common/include/rte_lcore.h
lrwxrwxrwx 1 root root    48 Feb 20 22:22 rte_log.h -> ../../../lib/librte_eal/common/include/rte_log.h
lrwxrwxrwx 1 root root    34 Feb 20 22:22 rte_lpm6.h -> ../../../lib/librte_lpm/rte_lpm6.h
lrwxrwxrwx 1 root root    33 Feb 20 22:22 rte_lpm.h -> ../../../lib/librte_lpm/rte_lpm.h
lrwxrwxrwx 1 root root    37 Feb 20 22:22 rte_lpm_sse.h -> ../../../lib/librte_lpm/rte_lpm_sse.h
lrwxrwxrwx 1 root root    35 Feb 20 22:22 rte_lru.h -> ../../../lib/librte_table/rte_lru.h
lrwxrwxrwx 1 root root    39 Feb 20 22:22 rte_lru_x86.h -> ../../../lib/librte_table/rte_lru_x86.h
lrwxrwxrwx 1 root root    51 Feb 20 22:22 rte_malloc.h -> ../../../lib/librte_eal/common/include/rte_malloc.h
lrwxrwxrwx 1 root root    56 Feb 20 22:22 rte_malloc_heap.h -> ../../../lib/librte_eal/common/include/rte_malloc_heap.h
lrwxrwxrwx 1 root root    35 Feb 20 22:22 rte_mbuf.h -> ../../../lib/librte_mbuf/rte_mbuf.h
lrwxrwxrwx 1 root root    41 Feb 20 22:22 rte_mbuf_ptype.h -> ../../../lib/librte_mbuf/rte_mbuf_ptype.h
lrwxrwxrwx 1 root root    60 Feb 20 22:22 rte_memcpy.h -> ../../../lib/librte_eal/common/include/arch/x86/rte_memcpy.h
lrwxrwxrwx 1 root root    51 Feb 20 22:22 rte_memory.h -> ../../../lib/librte_eal/common/include/rte_memory.h
lrwxrwxrwx 1 root root    41 Feb 20 22:22 rte_mempool.h -> ../../../lib/librte_mempool/rte_mempool.h
lrwxrwxrwx 1 root root    52 Feb 20 22:22 rte_memzone.h -> ../../../lib/librte_eal/common/include/rte_memzone.h
lrwxrwxrwx 1 root root    37 Feb 20 22:22 rte_meter.h -> ../../../lib/librte_meter/rte_meter.h
lrwxrwxrwx 1 root root    41 Feb 20 22:22 rte_metrics.h -> ../../../lib/librte_metrics/rte_metrics.h
lrwxrwxrwx 1 root root    37 Feb 20 22:22 rte_net_crc.h -> ../../../lib/librte_net/rte_net_crc.h
lrwxrwxrwx 1 root root    33 Feb 20 22:22 rte_net.h -> ../../../lib/librte_net/rte_net.h
lrwxrwxrwx 1 root root    59 Feb 20 22:22 rte_pause.h -> ../../../lib/librte_eal/common/include/arch/x86/rte_pause.h
lrwxrwxrwx 1 root root    65 Feb 20 22:22 rte_pci_dev_feature_defs.h -> ../../../lib/librte_eal/common/include/rte_pci_dev_feature_defs.h
lrwxrwxrwx 1 root root    61 Feb 20 22:22 rte_pci_dev_features.h -> ../../../lib/librte_eal/common/include/rte_pci_dev_features.h
lrwxrwxrwx 1 root root    48 Feb 20 22:22 rte_pci.h -> ../../../lib/librte_eal/common/include/rte_pci.h
lrwxrwxrwx 1 root root    37 Feb 20 22:22 rte_pdump.h -> ../../../lib/librte_pdump/rte_pdump.h
lrwxrwxrwx 1 root root    54 Feb 20 22:22 rte_per_lcore.h -> ../../../lib/librte_eal/common/include/rte_per_lcore.h
lrwxrwxrwx 1 root root    43 Feb 20 22:22 rte_pipeline.h -> ../../../lib/librte_pipeline/rte_pipeline.h
lrwxrwxrwx 1 root root    40 Feb 20 22:23 rte_pmd_bnxt.h -> ../../../drivers/net/bnxt/rte_pmd_bnxt.h
lrwxrwxrwx 1 root root    40 Feb 20 22:23 rte_pmd_i40e.h -> ../../../drivers/net/i40e/rte_pmd_i40e.h
lrwxrwxrwx 1 root root    42 Feb 20 22:23 rte_pmd_ixgbe.h -> ../../../drivers/net/ixgbe/rte_pmd_ixgbe.h
lrwxrwxrwx 1 root root    56 Feb 20 22:24 rte_pmd_octeontx_ssovf.h -> ../../../drivers/event/octeontx/rte_pmd_octeontx_ssovf.h
lrwxrwxrwx 1 root root    42 Feb 20 22:22 rte_port_ethdev.h -> ../../../lib/librte_port/rte_port_ethdev.h
lrwxrwxrwx 1 root root    38 Feb 20 22:22 rte_port_fd.h -> ../../../lib/librte_port/rte_port_fd.h
lrwxrwxrwx 1 root root    40 Feb 20 22:22 rte_port_frag.h -> ../../../lib/librte_port/rte_port_frag.h
lrwxrwxrwx 1 root root    35 Feb 20 22:22 rte_port.h -> ../../../lib/librte_port/rte_port.h
lrwxrwxrwx 1 root root    39 Feb 20 22:22 rte_port_kni.h -> ../../../lib/librte_port/rte_port_kni.h
lrwxrwxrwx 1 root root    39 Feb 20 22:22 rte_port_ras.h -> ../../../lib/librte_port/rte_port_ras.h
lrwxrwxrwx 1 root root    40 Feb 20 22:22 rte_port_ring.h -> ../../../lib/librte_port/rte_port_ring.h
lrwxrwxrwx 1 root root    41 Feb 20 22:22 rte_port_sched.h -> ../../../lib/librte_port/rte_port_sched.h
lrwxrwxrwx 1 root root    47 Feb 20 22:22 rte_port_source_sink.h -> ../../../lib/librte_port/rte_port_source_sink.h
lrwxrwxrwx 1 root root    37 Feb 20 22:22 rte_power.h -> ../../../lib/librte_power/rte_power.h
lrwxrwxrwx 1 root root    62 Feb 20 22:22 rte_prefetch.h -> ../../../lib/librte_eal/common/include/arch/x86/rte_prefetch.h
lrwxrwxrwx 1 root root    51 Feb 20 22:22 rte_random.h -> ../../../lib/librte_eal/common/include/rte_random.h
lrwxrwxrwx 1 root root    42 Feb 20 22:22 rte_reciprocal.h -> ../../../lib/librte_sched/rte_reciprocal.h
lrwxrwxrwx 1 root root    35 Feb 20 22:22 rte_red.h -> ../../../lib/librte_sched/rte_red.h
lrwxrwxrwx 1 root root    41 Feb 20 22:22 rte_reorder.h -> ../../../lib/librte_reorder/rte_reorder.h
lrwxrwxrwx 1 root root    35 Feb 20 22:22 rte_ring.h -> ../../../lib/librte_ring/rte_ring.h
lrwxrwxrwx 1 root root    57 Feb 20 22:22 rte_rtm.h -> ../../../lib/librte_eal/common/include/arch/x86/rte_rtm.h
lrwxrwxrwx 1 root root    60 Feb 20 22:22 rte_rwlock.h -> ../../../lib/librte_eal/common/include/arch/x86/rte_rwlock.h
lrwxrwxrwx 1 root root    44 Feb 20 22:22 rte_sched_common.h -> ../../../lib/librte_sched/rte_sched_common.h
lrwxrwxrwx 1 root root    37 Feb 20 22:22 rte_sched.h -> ../../../lib/librte_sched/rte_sched.h
lrwxrwxrwx 1 root root    34 Feb 20 22:22 rte_sctp.h -> ../../../lib/librte_net/rte_sctp.h
lrwxrwxrwx 1 root root    62 Feb 20 22:22 rte_service_component.h -> ../../../lib/librte_eal/common/include/rte_service_component.h
lrwxrwxrwx 1 root root    52 Feb 20 22:22 rte_service.h -> ../../../lib/librte_eal/common/include/rte_service.h
lrwxrwxrwx 1 root root    62 Feb 20 22:22 rte_spinlock.h -> ../../../lib/librte_eal/common/include/arch/x86/rte_spinlock.h
lrwxrwxrwx 1 root root    55 Feb 20 22:22 rte_string_fns.h -> ../../../lib/librte_eal/common/include/rte_string_fns.h
lrwxrwxrwx 1 root root    41 Feb 20 22:22 rte_table_acl.h -> ../../../lib/librte_table/rte_table_acl.h
lrwxrwxrwx 1 root root    43 Feb 20 22:22 rte_table_array.h -> ../../../lib/librte_table/rte_table_array.h
lrwxrwxrwx 1 root root    37 Feb 20 22:22 rte_table.h -> ../../../lib/librte_table/rte_table.h
lrwxrwxrwx 1 root root    42 Feb 20 22:22 rte_table_hash.h -> ../../../lib/librte_table/rte_table_hash.h
lrwxrwxrwx 1 root root    41 Feb 20 22:22 rte_table_lpm.h -> ../../../lib/librte_table/rte_table_lpm.h
lrwxrwxrwx 1 root root    46 Feb 20 22:22 rte_table_lpm_ipv6.h -> ../../../lib/librte_table/rte_table_lpm_ipv6.h
lrwxrwxrwx 1 root root    42 Feb 20 22:22 rte_table_stub.h -> ../../../lib/librte_table/rte_table_stub.h
lrwxrwxrwx 1 root root    50 Feb 20 22:22 rte_tailq.h -> ../../../lib/librte_eal/common/include/rte_tailq.h
lrwxrwxrwx 1 root root    33 Feb 20 22:22 rte_tcp.h -> ../../../lib/librte_net/rte_tcp.h
lrwxrwxrwx 1 root root    36 Feb 20 22:22 rte_thash.h -> ../../../lib/librte_hash/rte_thash.h
lrwxrwxrwx 1 root root    49 Feb 20 22:22 rte_time.h -> ../../../lib/librte_eal/common/include/rte_time.h
lrwxrwxrwx 1 root root    37 Feb 20 22:22 rte_timer.h -> ../../../lib/librte_timer/rte_timer.h
lrwxrwxrwx 1 root root    41 Feb 20 22:22 rte_tm_driver.h -> ../../../lib/librte_ether/rte_tm_driver.h
lrwxrwxrwx 1 root root    34 Feb 20 22:22 rte_tm.h -> ../../../lib/librte_ether/rte_tm.h
lrwxrwxrwx 1 root root    33 Feb 20 22:22 rte_udp.h -> ../../../lib/librte_net/rte_udp.h
lrwxrwxrwx 1 root root    49 Feb 20 22:22 rte_vdev.h -> ../../../lib/librte_eal/common/include/rte_vdev.h
lrwxrwxrwx 1 root root    58 Feb 20 22:22 rte_vect.h -> ../../../lib/librte_eal/common/include/arch/x86/rte_vect.h
lrwxrwxrwx 1 root root    52 Feb 20 22:22 rte_version.h -> ../../../lib/librte_eal/common/include/rte_version.h
lrwxrwxrwx 1 root root    37 Feb 20 22:22 rte_vhost.h -> ../../../lib/librte_vhost/rte_vhost.h

lib:

drwxr-xr-x 2 root root   4096 Feb 20 22:24 ./
drwxr-xr-x 7 root root   4096 Feb 20 22:22 ../
-rw-r--r-- 1 root root   1191 Feb 20 22:26 libdpdk.a
-rw-r--r-- 1 root root 122324 Feb 20 22:22 librte_acl.a
-rw-r--r-- 1 root root   3238 Feb 20 22:22 librte_bitratestats.a
-rw-r--r-- 1 root root   9780 Feb 20 22:22 librte_cfgfile.a
-rw-r--r-- 1 root root  61450 Feb 20 22:22 librte_cmdline.a
-rw-r--r-- 1 root root  43438 Feb 20 22:22 librte_cryptodev.a
-rw-r--r-- 1 root root  35626 Feb 20 22:22 librte_distributor.a
-rw-r--r-- 1 root root 362274 Feb 20 22:22 librte_eal.a
-rw-r--r-- 1 root root  39984 Feb 20 22:22 librte_efd.a
-rw-r--r-- 1 root root  91234 Feb 20 22:22 librte_ethdev.a
-rw-r--r-- 1 root root  37140 Feb 20 22:22 librte_eventdev.a
-rw-r--r-- 1 root root  12570 Feb 20 22:22 librte_gro.a
-rw-r--r-- 1 root root  74284 Feb 20 22:22 librte_hash.a
-rw-r--r-- 1 root root  47212 Feb 20 22:22 librte_ip_frag.a
-rw-r--r-- 1 root root   5458 Feb 20 22:22 librte_jobstats.a
-rw-r--r-- 1 root root  22046 Feb 20 22:22 librte_kni.a
-rw-r--r-- 1 root root   3994 Feb 20 22:22 librte_kvargs.a
-rw-r--r-- 1 root root  10286 Feb 20 22:22 librte_latencystats.a
-rw-r--r-- 1 root root  33166 Feb 20 22:22 librte_lpm.a
-rw-r--r-- 1 root root  22136 Feb 20 22:22 librte_mbuf.a
-rw-r--r-- 1 root root  26430 Feb 20 22:22 librte_mempool.a
-rw-r--r-- 1 root root   7224 Feb 20 22:22 librte_mempool_ring.a
-rw-r--r-- 1 root root   4186 Feb 20 22:22 librte_mempool_stack.a
-rw-r--r-- 1 root root   3626 Feb 20 22:22 librte_meter.a
-rw-r--r-- 1 root root   5124 Feb 20 22:22 librte_metrics.a
-rw-r--r-- 1 root root  19214 Feb 20 22:22 librte_net.a
-rw-r--r-- 1 root root  28976 Feb 20 22:22 librte_pdump.a
-rw-r--r-- 1 root root  26942 Feb 20 22:22 librte_pipeline.a
-rw-r--r-- 1 root root  30930 Feb 20 22:22 librte_pmd_af_packet.a
-rw-r--r-- 1 root root 105350 Feb 20 22:22 librte_pmd_ark.a
-rw-r--r-- 1 root root  46530 Feb 20 22:23 librte_pmd_avp.a
-rw-r--r-- 1 root root 216094 Feb 20 22:23 librte_pmd_bnxt.a
-rw-r--r-- 1 root root 144452 Feb 20 22:23 librte_pmd_bond.a
-rw-r--r-- 1 root root  73730 Feb 20 22:24 librte_pmd_crypto_scheduler.a
-rw-r--r-- 1 root root 169646 Feb 20 22:23 librte_pmd_cxgbe.a
-rw-r--r-- 1 root root 525352 Feb 20 22:23 librte_pmd_e1000.a
-rw-r--r-- 1 root root  69486 Feb 20 22:23 librte_pmd_ena.a
-rw-r--r-- 1 root root 190284 Feb 20 22:23 librte_pmd_enic.a
-rw-r--r-- 1 root root  72106 Feb 20 22:23 librte_pmd_failsafe.a
-rw-r--r-- 1 root root 178806 Feb 20 22:23 librte_pmd_fm10k.a
-rw-r--r-- 1 root root 825588 Feb 20 22:23 librte_pmd_i40e.a
-rw-r--r-- 1 root root 582742 Feb 20 22:23 librte_pmd_ixgbe.a
-rw-r--r-- 1 root root  12530 Feb 20 22:24 librte_pmd_kni.a
-rw-r--r-- 1 root root  95072 Feb 20 22:23 librte_pmd_lio.a
-rw-r--r-- 1 root root  41714 Feb 20 22:23 librte_pmd_nfp.a
-rw-r--r-- 1 root root  20714 Feb 20 22:23 librte_pmd_null.a
-rw-r--r-- 1 root root  18450 Feb 20 22:24 librte_pmd_null_crypto.a
-rw-r--r-- 1 root root  33230 Feb 20 22:24 librte_pmd_octeontx_ssovf.a
-rw-r--r-- 1 root root 609034 Feb 20 22:24 librte_pmd_qede.a
-rw-r--r-- 1 root root  22896 Feb 20 22:24 librte_pmd_ring.a
-rw-r--r-- 1 root root 418264 Feb 20 22:24 librte_pmd_sfc_efx.a
-rw-r--r-- 1 root root  14066 Feb 20 22:24 librte_pmd_skeleton_event.a
-rw-r--r-- 1 root root  54108 Feb 20 22:24 librte_pmd_sw_event.a
-rw-r--r-- 1 root root  95158 Feb 20 22:24 librte_pmd_tap.a
-rw-r--r-- 1 root root 120874 Feb 20 22:24 librte_pmd_thunderx_nicvf.a
-rw-r--r-- 1 root root  29044 Feb 20 22:24 librte_pmd_vhost.a
-rw-r--r-- 1 root root 134624 Feb 20 22:24 librte_pmd_virtio.a
-rw-r--r-- 1 root root  52754 Feb 20 22:24 librte_pmd_vmxnet3_uio.a
-rw-r--r-- 1 root root 154716 Feb 20 22:22 librte_port.a
-rw-r--r-- 1 root root  31884 Feb 20 22:22 librte_power.a
-rw-r--r-- 1 root root  14966 Feb 20 22:22 librte_reorder.a
-rw-r--r-- 1 root root   8842 Feb 20 22:22 librte_ring.a
-rw-r--r-- 1 root root  41108 Feb 20 22:22 librte_sched.a
-rw-r--r-- 1 root root 176040 Feb 20 22:22 librte_table.a
-rw-r--r-- 1 root root  11272 Feb 20 22:22 librte_timer.a
-rw-r--r-- 1 root root 140572 Feb 20 22:22 librte_vhost.a
vipinpv85 commented 5 years ago

if you have used make install T=x86_64-native-linuxapp-gcc O=x86_64-native-linuxapp-gcc, then ls $RTE_SDK/$RTE_TARGET/include/, ls $RTE_SDK/$RTE_TARGET/lib/ is the above is true true.

But you had started with dpdk-16.07 with make install T=x86_64-native-linuxapp-gcc.

Do you have issue with 'dpdk-17.08.2' with building suricata? I still have not received your terminal access to help you more.

vipinpv85 commented 5 years ago

Please check if you get error 

have 'dpdk-include-common.h:19:10: fatal error: rte_pci_dev_ids.h: No such file or directory
 #include "rte_pci_dev_ids.h"
          ^~~~~~~~~~~~~~~~~~~

These has been removed after DPDK 2.2

duanrui123456 commented 5 years ago

There is indeed this error, so which version of DPDK should I use?

This is a personal virtual machine that I can't remotely access

vipinpv85 commented 5 years ago

I have been parallel checking with both versions you have used. As I pointed out

  1. if you use DPDK 2.2 or 16 series you have "rte_pci_dev_ids.h"
  2. if you use DPDK 17 you do not have "rte_pci_dev_ids.h"

I check in my local machine for grep STAILQ_ENTRY /usr/include/sys/queue.h I do have definitions.

Since you have given incomplete logs, and not terminal to check my hands are tied down to limited assumptions for debugging on your eniviroment.

Once again I humbly request to share your terminal for easier debug

vipinpv85 commented 5 years ago

If you are able to find the definition force the suricata make file for `make CFLAGS="-g -I/usr/include/".

I still mention this is not suricata changes which causes the problem but environment,

duanrui123456 commented 5 years ago

I have checked this header file before, and it exists, as follows


#define STAILQ_HEAD_INITIALIZER(head)                                   \
        { NULL, &(head).stqh_first }

#define STAILQ_ENTRY(type)                                              \
struct {                                                                \
        struct type *stqe_next; /* next element */                      \
}
vipinpv85 commented 5 years ago

have you tried `make CFLAGS="-g -I/usr/include/".

Why is it difficult to get terminal debug? I have been requesting this multiple times. It looks environment settings issue

vipinpv85 commented 5 years ago

so here is my ten cents,

It is easier to debug if there is terminal access. If there is no terminal access it is difficult to debug.

I am interested to hear you update after make CFLAGS="-g -I/usr/include/

There is indeed this error, so which version of DPDK should I use?

using 17.08.2, I shared you will first hit the header file not found error. Hence the logs and version you shared is mismatched.

vipinpv85 commented 5 years ago

HI duanrui123456,

I have reched the code and compiled the folder with DPDK 16.07.2. As per my suggestion I also used 'make CFLAGS="/usr/include/"'

Both scenarios do not produce error for STAILQ_ENTRY

duanrui123456 commented 5 years ago

Under dpdk-17.08.2, the following error was reported when I executed the following command

make CFLAGS="-g -I/usr/include/"
In file included from util-dpdk-setup.c:8:0:
dpdk-include-common.h:19:29: fatal error: rte_pci_dev_ids.h: No such file or directory
compilation terminated.
vipinpv85 commented 5 years ago

have you read my comment 

I have been parallel checking with both versions you have used. As I pointed out

1. if you use DPDK 2.2 or 16 series you have "rte_pci_dev_ids.h"
2. if you use DPDK 17 you do not have "rte_pci_dev_ids.h"

I check in my local machine for `grep STAILQ_ENTRY /usr/include/sys/queue.h` I do have definitions.

Since you have given incomplete logs, and not terminal to check my hands are tied down to limited assumptions for debugging on your eniviroment.

Once again I humbly request to share your terminal for easier debug
vipinpv85 commented 5 years ago

I think you are not helping and supporting with right access. I think you are consuming my time without support.

vipinpv85 commented 5 years ago

I am closing this as it compiles with DPDK 'dpdk-stable-16.07.2'. Closing the issue as there no sufficent support or help.

Action will update readme for better clarity and debug

llyyrr commented 5 years ago

Hi, I'm compiled this package on Centos 7 with DPDK 17.11.5. When you compiling this package to Centos with DPSK 17.11.5 there are many errors associated with the discrepancy in the versions of the kernel and dpdk. I ask you to specify which version kernel and DPDK version to use?

vipinpv85 commented 5 years ago

Requesting to go through readme, for the version support.

With respect to dpdk build and kernel errors, without information I can not predict your problem or solution.

Kernel version please check with suricata and dpdk if they version specific.

Note: of you have bugs or issues please open it in the template .

vipinpv85 commented 5 years ago

updated with DPDK 17.11.3 with readme. try and let me know.

llyyrr commented 5 years ago

I successfully compiled Surucata with DPDK 17.11.5, but the launch of suricata fails:

suricata --dpdkintel -v

EAL: Detected 8 lcore(s) EAL: Probing VFIO support... EAL: VFIO support initialized EAL: PCI device 0000:02:00.0 on NUMA socket -1 EAL: Invalid NUMA socket, default to 0 EAL: probe driver: 8086:154d net_ixgbe EAL: PCI device 0000:02:00.1 on NUMA socket -1 EAL: Invalid NUMA socket, default to 0 EAL: probe driver: 8086:154d net_ixgbe 10/4/2019 -- 13:58:17 - - This is Suricata version 3.0 RELEASE 10/4/2019 -- 13:58:17 - - CPUs/cores online: 8 10/4/2019 -- 13:58:17 - - Adding interface 0000:02:00.0 from config file 10/4/2019 -- 13:58:17 - - Adding interface 0000:02:00.1 from config file 10/4/2019 -- 13:58:17 - - 'default' server has 'request-body-minimal-inspect-size' set to 33882 and 'request-body-inspect-window' set to 4053 after randomization. 10/4/2019 -- 13:58:17 - - 'default' server has 'response-body-minimal-inspect-size' set to 42119 and 'response-body-inspect-window' set to 16872 after randomization. 10/4/2019 -- 13:58:17 - - Protocol detection and parser disabled for smb protocol. 10/4/2019 -- 13:58:17 - - Protocol detection and parser disabled for dcerpc protocol. 10/4/2019 -- 13:58:17 - - Protocol detection and parser disabled for dcerpc protocol. 10/4/2019 -- 13:58:17 - - Protocol detection and parser disabled for smtp protocol. 10/4/2019 -- 13:58:17 - - DNS request flood protection level: 500 10/4/2019 -- 13:58:17 - - DNS per flow memcap (state-memcap): 524288 10/4/2019 -- 13:58:17 - - DNS global memcap: 16777216 10/4/2019 -- 13:58:17 - - Protocol detection and parser disabled for modbus protocol. 10/4/2019 -- 13:58:17 - - allocated 3670016 bytes of memory for the defrag hash... 65536 buckets of size 56 10/4/2019 -- 13:58:17 - - preallocated 65535 defrag trackers of size 168 10/4/2019 -- 13:58:17 - - defrag memory usage: 14679896 bytes, maximum: 33554432 10/4/2019 -- 13:58:17 - - AutoFP mode using default "Active Packets" flow load balancer 10/4/2019 -- 13:58:17 - - allocated 262144 bytes of memory for the host hash... 4096 buckets of size 64 10/4/2019 -- 13:58:17 - - preallocated 1000 hosts of size 136 10/4/2019 -- 13:58:17 - - host memory usage: 398144 bytes, maximum: 16777216 10/4/2019 -- 13:58:17 - - allocated 4194304 bytes of memory for the flow hash... 65536 buckets of size 64 10/4/2019 -- 13:58:17 - - preallocated 10000 flows of size 288 10/4/2019 -- 13:58:17 - - flow memory usage: 7074304 bytes, maximum: 1073741824 10/4/2019 -- 13:58:17 - - stream "prealloc-sessions": 2048 (per thread) 10/4/2019 -- 13:58:17 - - stream "memcap": 536870912 10/4/2019 -- 13:58:17 - - stream "midstream" session pickups: disabled 10/4/2019 -- 13:58:17 - - stream "async-oneside": disabled 10/4/2019 -- 13:58:17 - - stream "checksum-validation": enabled 10/4/2019 -- 13:58:17 - - stream."inline": disabled 10/4/2019 -- 13:58:17 - - stream "max-synack-queued": 5 10/4/2019 -- 13:58:17 - - stream.reassembly "memcap": 2147483648 10/4/2019 -- 13:58:17 - - stream.reassembly "depth": 1048576 10/4/2019 -- 13:58:17 - - stream.reassembly "toserver-chunk-size": 2551 10/4/2019 -- 13:58:17 - - stream.reassembly "toclient-chunk-size": 2650 10/4/2019 -- 13:58:17 - - stream.reassembly.raw: enabled 10/4/2019 -- 13:58:17 - - segment pool: pktsize 4, prealloc 256 10/4/2019 -- 13:58:17 - - segment pool: pktsize 16, prealloc 512 10/4/2019 -- 13:58:17 - - segment pool: pktsize 112, prealloc 512 10/4/2019 -- 13:58:17 - - segment pool: pktsize 248, prealloc 512 10/4/2019 -- 13:58:17 - - segment pool: pktsize 512, prealloc 512 10/4/2019 -- 13:58:17 - - segment pool: pktsize 768, prealloc 1024 10/4/2019 -- 13:58:17 - - segment pool: pktsize 1448, prealloc 1024 10/4/2019 -- 13:58:17 - - segment pool: pktsize 65535, prealloc 128 10/4/2019 -- 13:58:17 - - stream.reassembly "chunk-prealloc": 250 10/4/2019 -- 13:58:17 - - stream.reassembly "zero-copy-size": 128 10/4/2019 -- 13:58:17 - - allocated 262144 bytes of memory for the ippair hash... 4096 buckets of size 64 10/4/2019 -- 13:58:17 - - preallocated 1000 ippairs of size 136 10/4/2019 -- 13:58:17 - - ippair memory usage: 398144 bytes, maximum: 16777216 10/4/2019 -- 13:58:17 - - using magic-file /usr/share/file/magic 10/4/2019 -- 13:58:17 - - Delayed detect disabled 10/4/2019 -- 13:58:17 - - IP reputation disabled 10/4/2019 -- 13:58:17 - - Loading rule file: /usr/local/etc/suricata/rules/drop.rules 10/4/2019 -- 13:58:17 - - Loading rule file: /usr/local/etc/suricata/rules/emerging-dos.rules 10/4/2019 -- 13:58:17 - - Loading rule file: /usr/local/etc/suricata/rules/emerging-games.rules 10/4/2019 -- 13:58:17 - - Loading rule file: /usr/local/etc/suricata/rules/stream-events.rules 10/4/2019 -- 13:58:17 - - Loading rule file: /usr/local/etc/suricata/rules/dns-events.rules 10/4/2019 -- 13:58:17 - - 5 rule files processed. 165 rules successfully loaded, 0 rules failed 10/4/2019 -- 13:58:17 - - 165 signatures processed. 30 are IP-only rules, 57 are inspecting packet payload, 27 inspect application layer, 0 are decoder event only 10/4/2019 -- 13:58:17 - - building signature grouping structure, stage 1: preprocessing rules... complete 10/4/2019 -- 13:58:17 - - building signature grouping structure, stage 2: building source address list... complete 10/4/2019 -- 13:58:17 - - building signature grouping structure, stage 3: building destination address lists... complete 10/4/2019 -- 13:58:17 - - Threshold config parsed: 0 rule(s) found 10/4/2019 -- 13:58:17 - - Core dump size set to unlimited. 10/4/2019 -- 13:58:17 - - dropped the caps for main thread 10/4/2019 -- 13:58:17 - - fast output device (regular) initialized: fast.log 10/4/2019 -- 13:58:17 - - stats output device (regular) initialized: stats.log 10/4/2019 -- 13:58:17 - - [ERRCODE: SC_ERR_MISSING_CONFIG_PARAM(118)] - DPDK IOPL init -1 PANIC in dpdkConfSetup(): ERROR: Cannot init IOPL 6: [suricata() [0x458a50]] 5: [/lib64/libc.so.6(libc_start_main+0xf5) [0x7f42de24c3d5]] 4: [suricata(main+0x2153) [0x457a73]] 3: [suricata(dpdkConfSetup+0x49b) [0x5f326b]] 2: [suricata(rte_panic+0xb8) [0x455820]] 1: [suricata(rte_dump_stack+0x1a) [0x7253ba]] Aborted (core dumped)

In Linux man-pages it is written: "The I/O privilege level for a normal process is 0. This call is mostly for the i386 architecture. On many other architectures it does not exist or will always return an error."

vipinpv85 commented 5 years ago
  1. this does not look like orginal error. Hence it will be easier to track if it is opened new.
  2. I have tested only on x86_64 and not in i386.

for your particular can you try commenting rte_eal_iopl_init in dpdkConfSetup

llyyrr commented 5 years ago

I do not use ш386 architecture.

DPDK_TARGET = "x86_64-native-linuxapp-gcc"

uname -rm 4.4.178-1.el7.elrepo.x86_64 x86_64

vipinpv85 commented 5 years ago

Did you try rebuilding with 'commenting rte_eal_iopl_init in dpdkConfSetup'

vipinpv85 commented 5 years ago

rte_eal_iopl_init is used to get privellege access to ioports especially for virtio. Since both NIC you are using in ixgbe you are safe to comment

llyyrr commented 5 years ago

I found this function only in the util-dpdk-setup.c file.

vipinpv85 commented 5 years ago

I am not sure what is that I am not communicating clearly. Let me try again.

steps to follow

  1. comment the code or return check for rte_eal_iopl_init in dpdkConfSetup.
  2. rebuild suricata for your environment.
  3. run and test for your environment.
vipinpv85 commented 5 years ago

I found this function only in the util-dpdk-setup.c file.

I am waiting for your update.

llyyrr commented 5 years ago

After recovering Suricata an error occurred when starting Suricata: 11/4/2019 -- 10:04:09 - - This is Suricata version 3.0 RELEASE 11/4/2019 -- 10:04:09 - - ----- Global DPDK-INTEL Config ----- 11/4/2019 -- 10:04:09 - - Number Of Ports : 2 11/4/2019 -- 10:04:09 - - Operation Mode : IPS 11/4/2019 -- 10:04:09 - - Port:0, Map:1 11/4/2019 -- 10:04:09 - - Port:1, Map:0 11/4/2019 -- 10:04:09 - - ------------------------------------ 11/4/2019 -- 10:04:09 - - [ERRCODE: SC_ERR_DPDKINTEL_MEM_FAILED(276)] - mbuf_pool alloc failed 11/4/2019 -- 10:04:09 - - [ERRCODE: SC_ERR_MISSING_CONFIG_PARAM(118)] - DPDK dev setup!!!

AnonHugePages: 8192 kB HugePages_Total: 32 HugePages_Free: 0 HugePages_Rsvd: 0 HugePages_Surp: 0 Hugepagesize: 1048576 kB

I was this problem before, but I do not remember the solution to this problem(((.

vipinpv85 commented 5 years ago

So after commenting and rebuilding it works, now you are faced with mbuf_poll alloc failed, since 'HugePages_Free: 0'.

easiest way is to rm /mnt/huge/rte_* and try again

llyyrr commented 5 years ago

Hi, I replaced the rte_mempool_create function in dpdkConfSetup on the rte_pktmbuf_pool_create. Now I are faced with Unknown runtime mode. Run Log Suricata:

suricata --dpdkintel -c /usr/local/etc/suricata/suricata.yaml --runmode=workers -v

EAL: Detected 8 lcore(s) EAL: Probing VFIO support... EAL: VFIO support initialized EAL: PCI device 0000:02:00.0 on NUMA socket -1 EAL: Invalid NUMA socket, default to 0 EAL: probe driver: 8086:154d net_ixgbe EAL: PCI device 0000:02:00.1 on NUMA socket -1 EAL: Invalid NUMA socket, default to 0 EAL: probe driver: 8086:154d net_ixgbe 16/4/2019 -- 15:31:22 - - This is Suricata version 3.0 RELEASE 16/4/2019 -- 15:31:22 - - CPUs/cores online: 8 16/4/2019 -- 15:31:22 - - Adding interface 0000:02:00.0 from config file 16/4/2019 -- 15:31:22 - - Adding interface 0000:02:00.1 from config file 16/4/2019 -- 15:31:22 - - 'default' server has 'request-body-minimal-inspect-size' set to 33882 and 'request-body-inspect-window' set to 4053 after randomization. 16/4/2019 -- 15:31:22 - - 'default' server has 'response-body-minimal-inspect-size' set to 42119 and 'response-body-inspect-window' set to 16872 after randomization. 16/4/2019 -- 15:31:22 - - Protocol detection and parser disabled for smb protocol. 16/4/2019 -- 15:31:22 - - Protocol detection and parser disabled for dcerpc protocol. 16/4/2019 -- 15:31:22 - - Protocol detection and parser disabled for dcerpc protocol. 16/4/2019 -- 15:31:22 - - Protocol detection and parser disabled for smtp protocol. 16/4/2019 -- 15:31:22 - - DNS request flood protection level: 500 16/4/2019 -- 15:31:22 - - DNS per flow memcap (state-memcap): 524288 16/4/2019 -- 15:31:22 - - DNS global memcap: 16777216 16/4/2019 -- 15:31:22 - - Protocol detection and parser disabled for modbus protocol. 16/4/2019 -- 15:31:22 - - allocated 3670016 bytes of memory for the defrag hash... 65536 buckets of size 56 16/4/2019 -- 15:31:22 - - preallocated 65535 defrag trackers of size 168 16/4/2019 -- 15:31:22 - - defrag memory usage: 14679896 bytes, maximum: 33554432 16/4/2019 -- 15:31:22 - - AutoFP mode using default "Active Packets" flow load balancer 16/4/2019 -- 15:31:22 - - allocated 262144 bytes of memory for the host hash... 4096 buckets of size 64 16/4/2019 -- 15:31:22 - - preallocated 1000 hosts of size 136 16/4/2019 -- 15:31:22 - - host memory usage: 398144 bytes, maximum: 16777216 16/4/2019 -- 15:31:22 - - allocated 4194304 bytes of memory for the flow hash... 65536 buckets of size 64 16/4/2019 -- 15:31:22 - - preallocated 10000 flows of size 288 16/4/2019 -- 15:31:22 - - flow memory usage: 7074304 bytes, maximum: 1073741824 16/4/2019 -- 15:31:22 - - stream "prealloc-sessions": 2048 (per thread) 16/4/2019 -- 15:31:22 - - stream "memcap": 536870912 16/4/2019 -- 15:31:22 - - stream "midstream" session pickups: disabled 16/4/2019 -- 15:31:22 - - stream "async-oneside": disabled 16/4/2019 -- 15:31:22 - - stream "checksum-validation": enabled 16/4/2019 -- 15:31:22 - - stream."inline": disabled 16/4/2019 -- 15:31:22 - - stream "max-synack-queued": 5 16/4/2019 -- 15:31:22 - - stream.reassembly "memcap": 2147483648 16/4/2019 -- 15:31:22 - - stream.reassembly "depth": 1048576 16/4/2019 -- 15:31:22 - - stream.reassembly "toserver-chunk-size": 2574 16/4/2019 -- 15:31:22 - - stream.reassembly "toclient-chunk-size": 2646 16/4/2019 -- 15:31:22 - - stream.reassembly.raw: enabled 16/4/2019 -- 15:31:22 - - segment pool: pktsize 4, prealloc 256 16/4/2019 -- 15:31:22 - - segment pool: pktsize 16, prealloc 512 16/4/2019 -- 15:31:22 - - segment pool: pktsize 112, prealloc 512 16/4/2019 -- 15:31:22 - - segment pool: pktsize 248, prealloc 512 16/4/2019 -- 15:31:22 - - segment pool: pktsize 512, prealloc 512 16/4/2019 -- 15:31:22 - - segment pool: pktsize 768, prealloc 1024 16/4/2019 -- 15:31:22 - - segment pool: pktsize 1448, prealloc 1024 16/4/2019 -- 15:31:22 - - segment pool: pktsize 65535, prealloc 128 16/4/2019 -- 15:31:22 - - stream.reassembly "chunk-prealloc": 250 16/4/2019 -- 15:31:22 - - stream.reassembly "zero-copy-size": 128 16/4/2019 -- 15:31:22 - - allocated 262144 bytes of memory for the ippair hash... 4096 buckets of size 64 16/4/2019 -- 15:31:22 - - preallocated 1000 ippairs of size 136 16/4/2019 -- 15:31:22 - - ippair memory usage: 398144 bytes, maximum: 16777216 16/4/2019 -- 15:31:22 - - using magic-file /usr/share/file/magic 16/4/2019 -- 15:31:22 - - Delayed detect disabled 16/4/2019 -- 15:31:22 - - IP reputation disabled 16/4/2019 -- 15:31:22 - - Loading rule file: /usr/local/etc/suricata/rules/drop.rules 16/4/2019 -- 15:31:22 - - Loading rule file: /usr/local/etc/suricata/rules/emerging-dos.rules 16/4/2019 -- 15:31:22 - - Loading rule file: /usr/local/etc/suricata/rules/emerging-games.rules 16/4/2019 -- 15:31:22 - - Loading rule file: /usr/local/etc/suricata/rules/stream-events.rules 16/4/2019 -- 15:31:22 - - Loading rule file: /usr/local/etc/suricata/rules/dns-events.rules 16/4/2019 -- 15:31:22 - - 5 rule files processed. 166 rules successfully loaded, 0 rules failed 16/4/2019 -- 15:31:22 - - 166 signatures processed. 31 are IP-only rules, 57 are inspecting packet payload, 27 inspect application layer, 0 are decoder event only 16/4/2019 -- 15:31:22 - - building signature grouping structure, stage 1: preprocessing rules... complete 16/4/2019 -- 15:31:22 - - building signature grouping structure, stage 2: building source address list... complete 16/4/2019 -- 15:31:22 - - building signature grouping structure, stage 3: building destination address lists... complete 16/4/2019 -- 15:31:22 - - Threshold config parsed: 0 rule(s) found 16/4/2019 -- 15:31:22 - - Core dump size set to unlimited. 16/4/2019 -- 15:31:22 - - dropped the caps for main thread 16/4/2019 -- 15:31:22 - - fast output device (regular) initialized: fast.log 16/4/2019 -- 15:31:22 - - stats output device (regular) initialized: stats.log 16/4/2019 -- 15:31:22 - - ----- Global DPDK-INTEL Config ----- 16/4/2019 -- 15:31:22 - - Number Of Ports : 2 16/4/2019 -- 15:31:22 - - Operation Mode : IPS 16/4/2019 -- 15:31:22 - - Port:0, Map:1 16/4/2019 -- 15:31:22 - - Port:1, Map:0 16/4/2019 -- 15:31:22 - - ------------------------------------ 16/4/2019 -- 15:31:23 - - DPDK OPMODE set to IPS!!! 16/4/2019 -- 15:31:23 - - DPDK OPMODE set to 2!!! 16/4/2019 -- 15:31:23 - - ----- Match Pattern ---- 16/4/2019 -- 15:31:23 - - http: 0 16/4/2019 -- 15:31:23 - - ftp: 0 16/4/2019 -- 15:31:23 - - tls: 0 16/4/2019 -- 15:31:23 - - dns: 0 16/4/2019 -- 15:31:23 - - smtp: 0 16/4/2019 -- 15:31:23 - - ssh: 0 16/4/2019 -- 15:31:23 - - smb: 0 16/4/2019 -- 15:31:23 - - smb2: 0 16/4/2019 -- 15:31:23 - - dcerpc:0 16/4/2019 -- 15:31:23 - - tcp: 0 16/4/2019 -- 15:31:23 - - udp: 0 16/4/2019 -- 15:31:23 - - sctp: 0 16/4/2019 -- 15:31:23 - - icmpv6:0 16/4/2019 -- 15:31:23 - - gre: 0 16/4/2019 -- 15:31:23 - - raw: 0 16/4/2019 -- 15:31:23 - - ipv4: 0 16/4/2019 -- 15:31:23 - - * ipv6: 0 16/4/2019 -- 15:31:23 - - ----------------------- 16/4/2019 -- 15:31:23 - - [ERRCODE: SC_ERR_UNKNOWN_RUN_MODE(125)] - Unknown runtime mode. Aborting

vipinpv85 commented 5 years ago

A couple of points to highlight

    • rte_mempool_create is used in dpdkIntelDevSetup and not in dpdkConfSetup.
    • DPDK function rte_pktmbuf_pool_create creates mempool objects with type cast as rte_mbuf (and with prvate_data which is not used in current solution).
    • Operation mode is selcted as 'Operation Mode : IPS'

Requested action items for

  1. Issue 4 is different from the reported error. Please open a new issue.
  2. Please share the command line and use the template to fill in the details.
  3. My last commit is '0186d7c60b4c668f5d0d3ec534e714e18cde45f8' with 'src: disabled panic and error for rte_eal_iopl_init'. What is your git log?
  4. Only explanation is if RunModeDpdkIntelGetDefaultMode is not invoked in runmodes.c. I think you have not updated to current version
vipinpv85 commented 5 years ago

I have not received any updates which look like the issue is due not updating code base. I have also not found any new error posted hence I am assuming there is no issue.

ahubaoan commented 5 years ago

I think I have encountered the same problem with the author and solved it. [VV] Are refering author as one who has created the issue ticket? So I do not understand what are you solving.

The reason is that queue.h is used in the suricata source code. It should be copied from the system /usr/include/sys/queue.h. Then it is suricata issue and not dpdk code change

They have a common macro, which can only be included once:

#ifndef _SYS_QUEUE_H_
#define _SYS_QUEUE_H_

I think I can try to fix this problem by adding the following in source-dpdk.h: [VV] in the current code there are no files as mentioned by you. But source-dpdkintel.h

#ifdef _SYS_QUEUE_H_
#undef _SYS_QUEUE_H_
#include <sys/queue.h>
#endif

Are you master branch? If yes, share you config.log and steps you have followed

ahubaoan please update your progress for task on DPDK-Suricata-4.1.1

vipinpv85 commented 4 years ago

please share the update for sys_queue with the master branch.

As like earlier cases, there are no responses when asked for

  1. Master branch
  2. Use the issue template for reporting.
  3. Share config file and error logs.

I find this really difficult to support such request without information.

vipinpv85 commented 4 years ago

@ahubaoan please share your updates and inputs. If you are not interested to share also, please do let me know so I can stop tracking this.

llyyrr commented 4 years ago

Hello! After a long time, I returned to this project. This time I managed to compile the program, but it does not work in IPS mode. Packets do not get into the ring buffer, and accordingly, they are not transferred to another port. If I turn off (comment) all rule set, then in IPS mode all packets are transmitted. Screenshot in the attachment. enabled rules: screen1 screen2 disabled rules: screen3 screen4

llyyrr commented 4 years ago

gcc version 7.4.0 (Ubuntu 7.4.0-1ubuntu1~18.04.1) DPDK Version: DPDK 17.11.3

DISTRIB_ID=Ubuntu DISTRIB_RELEASE=18.04 DISTRIB_CODENAME=bionic DISTRIB_DESCRIPTION="Ubuntu 18.04.2 LTS"

vipinpv85 commented 4 years ago

Hello! After a long time, I returned to this project. This time I managed to compile the program, but it does not work in IPS mode. Packets do not get into the ring buffer, and accordingly, they are not transferred to another port. If I turn off (comment) all rule set, then in IPS mode all packets are transmitted. Screenshot in the attachment. enabled rules: screen1 screen2 disabled rules: screen3 screen4

What is git commit tag? Are you sending traffic matching to the rules?

llyyrr commented 4 years ago

commit tag 156239ce9bf410107c9946b3f502106d73128598. I sending traffic that does not matching the rules! A set of rules consists of one rule that does not match the transmitted traffic.

vipinpv85 commented 4 years ago

I am unable to make out if you are using ReceiveDpdkPkts_IPS_1000 or ReceiveDpdkPkts_IPS_10000. Moreover DPDK port stats are showing for no rule match on port 1 you RX 40 and port 3 you TX 40 while on port 3 you RX 36 and port 1 you TX 36. This means since there is no rules configured in IPS DPDK thread works in bypass mode which is correct.

In rules configured mode, the RX stats are present but not TX. As this could be over site from my end of not printing DPDK stats, you can rely on dpdk-procinfo utility to check if packets are spit out.

If you still face issue with TX packets, I am open for a live debug to understand what is going wrong.

There are new features like ini, pre-parser, ACL and better stats logic been added in the latest release. If you are interested try the latest git commit from master branch.

llyyrr commented 4 years ago

In the rules mode, the packets is not transmitted in both directions. In the bypass mode, all packets are transmitted in both directions.

vipinpv85 commented 4 years ago

In the rules mode, the packets is not transmitted in both directions. In the bypass mode, all packets are transmitted in both directions.

the error is only 1 interface in IPS is reading packet and processing while other is dormant. I can check this, but as you have not shared for live debug or the configuration (100MB, 1g, 10G) this will take time from my end. But thanks for sharing this information.