search

marty90 / DPDK-Dump

DPDK-Dump is able to store on disk network traffic at high speed using DPDK library.
GNU General Public License v2.0
32 stars 28 forks source link

error when compile DPDK-Dump #5

Closed Trz123 closed 7 years ago

Trz123 commented 8 years ago

Hi, I got an error when I make the programe:

[root@localhost DPDK-Dump]# make CC src/main_dump.o cc1: warnings being treated as errors /home/dpdkinstall/DPDK-Dump/src/main_dump.c: In function ‘main_loop_consumer’: /home/dpdkinstall/dpdk-1.8.0/x86_64-native-linuxapp-gcc/include/rte_ring.h:628: error: dereferencing pointer ‘m.147’ does break strict-aliasing rules /home/dpdkinstall/dpdk-1.8.0/x86_64-native-linuxapp-gcc/include/rte_ring.h:703: error: dereferencing pointer ‘m.147’ does break strict-aliasing rules /home/dpdkinstall/DPDK-Dump/src/main_dump.c:183: note: initialized from here /home/dpdkinstall/dpdk-1.8.0/x86_64-native-linuxapp-gcc/include/rte_ring.h:703: error: dereferencing pointer ‘({anonymous})’ does break strict-aliasing rules /home/dpdkinstall/dpdk-1.8.0/x86_64-native-linuxapp-gcc/include/rte_ring.h:628: error: dereferencing pointer ‘({anonymous})’ does break strict-aliasing rules /home/dpdkinstall/DPDK-Dump/src/main_dump.c:183: note: initialized from here make[1]: * [src/main_dump.o] Error 1 make: * [all] Error 2

I use DPDK-1.8.0 and in centos6 x64

HaochuanXJTU commented 8 years ago

I have a same problem in centos6.7，but in ubuntu the programe can be built sucessful

marty90 commented 8 years ago

It seems a bug of dpdk with centos

HaochuanXJTU commented 8 years ago

Hi,do have demo to combine DPDK-DUMP to DPI?

marty90 commented 8 years ago

Hi, I combined DPDK-DUMP architecture with Tstat, an advanced passive monitoring tool. It collects hundreds of statistics, including DPI; Tstat-DPDK is able to scale up to 40Gbps, if on proper hardware. You find it here For information on the the architecture read this.

Trz123 commented 8 years ago

Thanks, I'm learning tstat

HaochuanXJTU commented 8 years ago

hi， I have install the Tstat-DPDK，but it seems some problem like this EAL: Detected lcore 0 as core 0 on socket 0 EAL: Detected lcore 1 as core 1 on socket 0 EAL: Detected lcore 2 as core 0 on socket 0 EAL: Detected lcore 3 as core 1 on socket 0 EAL: Support maximum 128 logical core(s) by configuration. EAL: Detected 4 lcore(s) EAL: cannot open VFIO container, error 2 (No such file or directory) EAL: VFIO support could not be initialized EAL: Setting up memory... EAL: Ask a virtual area of 0x200000 bytes EAL: Virtual area found at 0x7ffff1800000 (size = 0x200000) EAL: Ask a virtual area of 0x33800000 bytes EAL: Virtual area found at 0x7fffbde00000 (size = 0x33800000) EAL: Ask a virtual area of 0x55200000 bytes EAL: Virtual area found at 0x7fff68a00000 (size = 0x55200000) EAL: Ask a virtual area of 0x400000 bytes EAL: Virtual area found at 0x7fff68400000 (size = 0x400000) EAL: Ask a virtual area of 0x200000 bytes EAL: Virtual area found at 0x7fff68000000 (size = 0x200000) EAL: Ask a virtual area of 0x200000 bytes EAL: Virtual area found at 0x7fff67c00000 (size = 0x200000) EAL: Ask a virtual area of 0x200000 bytes EAL: Virtual area found at 0x7fff67800000 (size = 0x200000) EAL: Ask a virtual area of 0x200000 bytes EAL: Virtual area found at 0x7fff67400000 (size = 0x200000) EAL: Ask a virtual area of 0x600000 bytes EAL: Virtual area found at 0x7fff66c00000 (size = 0x600000) EAL: Ask a virtual area of 0x600000 bytes EAL: Virtual area found at 0x7fff66400000 (size = 0x600000) EAL: Ask a virtual area of 0x200000 bytes EAL: Virtual area found at 0x7fff66000000 (size = 0x200000) EAL: Ask a virtual area of 0x200000 bytes EAL: Virtual area found at 0x7fff65c00000 (size = 0x200000) EAL: Ask a virtual area of 0x200000 bytes EAL: Virtual area found at 0x7fff65800000 (size = 0x200000) EAL: Ask a virtual area of 0x200000 bytes EAL: Virtual area found at 0x7fff65400000 (size = 0x200000) EAL: Ask a virtual area of 0x600000 bytes EAL: Virtual area found at 0x7fff64c00000 (size = 0x600000) EAL: Ask a virtual area of 0x400000 bytes EAL: Virtual area found at 0x7fff64600000 (size = 0x400000) EAL: Ask a virtual area of 0x200000 bytes EAL: Virtual area found at 0x7fff64200000 (size = 0x200000) EAL: Ask a virtual area of 0x2200000 bytes EAL: Virtual area found at 0x7fff61e00000 (size = 0x2200000) EAL: Ask a virtual area of 0x200000 bytes EAL: Virtual area found at 0x7fff61a00000 (size = 0x200000) EAL: Ask a virtual area of 0xa00000 bytes EAL: Virtual area found at 0x7fff60e00000 (size = 0xa00000) EAL: Ask a virtual area of 0x1400000 bytes EAL: Virtual area found at 0x7fff5f800000 (size = 0x1400000) EAL: Ask a virtual area of 0x400000 bytes EAL: Virtual area found at 0x7fff5f200000 (size = 0x400000) EAL: Ask a virtual area of 0x400000 bytes EAL: Virtual area found at 0x7fff5ec00000 (size = 0x400000) EAL: Ask a virtual area of 0x200000 bytes EAL: Virtual area found at 0x7fff5e800000 (size = 0x200000) EAL: Ask a virtual area of 0x800000 bytes EAL: Virtual area found at 0x7fff5de00000 (size = 0x800000) EAL: Ask a virtual area of 0x200000 bytes EAL: Virtual area found at 0x7fff5da00000 (size = 0x200000) EAL: Ask a virtual area of 0x200000 bytes EAL: Virtual area found at 0x7fff5d600000 (size = 0x200000) EAL: Ask a virtual area of 0x200000 bytes EAL: Virtual area found at 0x7fff5d200000 (size = 0x200000) EAL: Ask a virtual area of 0x27400000 bytes EAL: Virtual area found at 0x7fff35c00000 (size = 0x27400000) EAL: Ask a virtual area of 0x200000 bytes EAL: Virtual area found at 0x7fff35800000 (size = 0x200000) EAL: Ask a virtual area of 0xa00000 bytes EAL: Virtual area found at 0x7fff34c00000 (size = 0xa00000) EAL: Ask a virtual area of 0x1c00000 bytes EAL: Virtual area found at 0x7fff32e00000 (size = 0x1c00000) EAL: Ask a virtual area of 0x400000 bytes EAL: Virtual area found at 0x7fff32800000 (size = 0x400000) EAL: Requesting 1497 pages of size 2MB from socket 0 EAL: TSC frequency is ~3093054 KHz EAL: Master core 0 is ready (tid=f7fcfa00) PMD: ENICPMD trace: rte_enic_pmd_init EAL: PCI device 0000:02:01.0 on NUMA socket -1 EAL: probe driver: 8086:100f rte_em_pmd EAL: 0000:02:01.0 not managed by UIO driver, skipping EAL: PCI device 0000:02:02.0 on NUMA socket -1 EAL: probe driver: 8086:100f rte_em_pmd EAL: PCI memory mapped at 0x7ffff1a00000 EAL: PCI memory mapped at 0x7ffff1a20000 PMD: eth_em_dev_init(): port_id 0 vendorID=0x8086 deviceID=0x100f EAL: PCI device 0000:02:03.0 on NUMA socket -1 EAL: probe driver: 8086:100f rte_em_pmd EAL: PCI memory mapped at 0x7ffff1a30000 EAL: PCI memory mapped at 0x7ffff1a50000 PMD: eth_em_dev_init(): port_id 1 vendorID=0x8086 deviceID=0x100f EAL: PCI device 0000:02:08.0 on NUMA socket -1 EAL: probe driver: 8086:100f rte_em_pmd EAL: PCI memory mapped at 0x7ffff1a60000 EAL: PCI memory mapped at 0x7ffff1a80000 PMD: eth_em_dev_init(): port_id 2 vendorID=0x8086 deviceID=0x100f EAL: PCI device 0000:02:09.0 on NUMA socket -1 EAL: probe driver: 8086:100f rte_em_pmd EAL: PCI memory mapped at 0x7ffff1a90000 EAL: PCI memory mapped at 0x7ffff1ab0000 PMD: eth_em_dev_init(): port_id 3 vendorID=0x8086 deviceID=0x100f [-] Disabling histo engine logs [-] Disabling log_skype_complete [-] Disabling log_chat_complete [-] Disabling log_chat_messages [-] Enabling log_http_complete [-] TCP log level set to 15 (Core + End_to_end + Layer7 + P2P + Options) [-] Video log level set to 31 (Core + End_to_end + Layer7 + VideoInfo + YouTube + Options) (-) Enabling full HTTP URL log [-] Disabling dump engine [Wed May 4 19:24:33 2016] created new outdir tstat_log00/2016_05_04_19_24.out

and the log is: log_http_complete log_mm_complete log_tcp_complete log_tcp_nocomplete log_udp_complete log_video_complete .swp ~

marty90 commented 8 years ago

this is the correct output. Tstat creates a set of files where it logs the activity on the network. Here you find information on the log files.

HaochuanXJTU commented 8 years ago

How can can i get the output like this: Instance: 0 Avg: 1.023us Max: 35.127us StdDev: 14.452 TCP cl.: 42 UDP cl.: 213 Rate: 0.897Mpps, Loss: 0Mpps Buffer occupation: 0% Instance: 1 Avg: 1.045us Max: 44.345us StdDev: 45.713 TCP cl.: 37 UDP cl.: 175 Rate: 0.813Mpps, Loss: 0Mpps Buffer occupation: 0%

marty90 commented 8 years ago

You must generate some traffic to the machine

HaochuanXJTU commented 8 years ago

ok,I see. Thank you！

HaochuanXJTU commented 8 years ago

I want to combine this DPDK-DUMP with nDPI,do you have have some advice?

marty90 commented 8 years ago

Tstat and nDPI are quite slow; actually nDPI is much slower than Tstat since it performs a deeper analysis. In Tstat-DPDK we solved by parallelizing on multiple cores the workload; on the contrary DPDK-Dump uses always two cores. It's mandatory to use at least 2 cores since one core must be dedicated to poll the network interfaces very frequently, while the other (dedicated to DPI) can be blocked due to disk I/O. If you don't need high performance, you can use DPDK Dump architecture, with 2 cores. If you want Nx10Gbps performance, you should use Tstat-DPDK architecture that can scale on multicore CPUs.

HaochuanXJTU commented 8 years ago

Excuse me for interrupting you，I have enconter some problem in configure queues，does it is the interface problem，it depend on NIC？

nb_sys_cores : 2 Name:rte_em_pmd Driver name: rte_em_pmd Max rx queues: 1 Max tx queues: 1 PCI Adress: 0000:02:02.0 EAL: Error - exiting with code: 1 Cause: Every interface must have a queue on each core, but this is not supported. Quitting...

marty90 commented 8 years ago

which network card are you using? i tested with intel 82599

HaochuanXJTU commented 8 years ago

I have solved the NIC problem by using a 82599 card, but there is another problem in my cpu, like this: mount: cpuset already mounted or /dev/cpuset busy mount: according to mtab, cpuset is already mounted on /dev/cpuset sh: 1: cannot create /dev/cpuset/tstat-dpdk01/cpuset.cpus: Permission denied sh: 1: cannot create /dev/cpuset/tstat-dpdk01/cpuset.mems: Permission denied sh: 1: cannot create /dev/cpuset/cpuset.sched_load_balance: Permission denied sh: 1: cannot create /dev/cpuset/cpuset.cpu_exclusive: Permission denied /bin/echo: write error: No space left on device

and this is my CPU information: processor : 0 vendor_id : GenuineIntel cpu family : 6 model : 23 model name : Intel(R) Core(TM)2 Duo CPU E7500 @ 2.93GHz stepping : 10 microcode : 0xa07 cpu MHz : 2936.000 cache size : 3072 KB physical id : 0 siblings : 2 core id : 0 cpu cores : 2 apicid : 0 initial apicid : 0 fpu : yes fpu_exception : yes cpuid level : 13 wp : yes flags : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx lm constant_tsc arch_perfmon pebs bts rep_good nopl aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm sse4_1 xsave lahf_lm dtherm tpr_shadow vnmi flexpriority bogomips : 5851.86 clflush size : 64 cache_alignment : 64 address sizes : 36 bits physical, 48 bits virtual power management:

processor : 1 vendor_id : GenuineIntel cpu family : 6 model : 23 model name : Intel(R) Core(TM)2 Duo CPU E7500 @ 2.93GHz stepping : 10 microcode : 0xa07 cpu MHz : 1603.000 cache size : 3072 KB physical id : 0 siblings : 2 core id : 1 cpu cores : 2 apicid : 1 initial apicid : 1 fpu : yes fpu_exception : yes cpuid level : 13 wp : yes flags : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx lm constant_tsc arch_perfmon pebs bts rep_good nopl aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm sse4_1 xsave lahf_lm dtherm tpr_shadow vnmi flexpriority bogomips : 5851.86 clflush size : 64 cache_alignment : 64 address sizes : 36 bits physical, 48 bits virtual power management:

Can I solve this problem by using a Server with a CPU E5?

marty90 commented 8 years ago

the CPU model doesn't matter. Do you sudo installed ? the program crashes or the execution continues?

HaochuanXJTU commented 8 years ago

The program crashes, sometimes it will show : Cannot set thread scheduling policy. what should be installed in my cumputer？

marty90 commented 8 years ago

As reported in the readme, you need a Linux Kernel >= 3.14, since we use SCHED_DEADLINE feature to speed performance. We tested on Debian and Ubuntu.

HaochuanXJTU commented 8 years ago

OK，thank you, my version is 3.13. I will update my kernel.

HaochuanXJTU commented 8 years ago

Hi, Which Linux Distribution do you use in testing Tstat-DPDK?

marty90 commented 8 years ago

Debian

HaochuanXJTU commented 8 years ago

I have test in my server，it's have a lot Loss.I need help

root@HaoChuan:/home/xjtu/tstat-dpdk# ./build/tstat_dpdk -c 1 -n 4 -- -m 2 EAL: Detected lcore 0 as core 0 on socket 0 EAL: Detected lcore 1 as core 1 on socket 0 EAL: Detected lcore 2 as core 2 on socket 0 EAL: Detected lcore 3 as core 3 on socket 0 EAL: Detected lcore 4 as core 4 on socket 0 EAL: Detected lcore 5 as core 5 on socket 0 EAL: Detected lcore 6 as core 0 on socket 1 EAL: Detected lcore 7 as core 1 on socket 1 EAL: Detected lcore 8 as core 2 on socket 1 EAL: Detected lcore 9 as core 3 on socket 1 EAL: Detected lcore 10 as core 4 on socket 1 EAL: Detected lcore 11 as core 5 on socket 1 EAL: Detected lcore 12 as core 0 on socket 0 EAL: Detected lcore 13 as core 1 on socket 0 EAL: Detected lcore 14 as core 2 on socket 0 EAL: Detected lcore 15 as core 3 on socket 0 EAL: Detected lcore 16 as core 4 on socket 0 EAL: Detected lcore 17 as core 5 on socket 0 EAL: Detected lcore 18 as core 0 on socket 1 EAL: Detected lcore 19 as core 1 on socket 1 EAL: Detected lcore 20 as core 2 on socket 1 EAL: Detected lcore 21 as core 3 on socket 1 EAL: Detected lcore 22 as core 4 on socket 1 EAL: Detected lcore 23 as core 5 on socket 1 EAL: Support maximum 128 logical core(s) by configuration. EAL: Detected 24 lcore(s) EAL: unsupported IOMMU type! EAL: VFIO support could not be initialized EAL: Setting up memory... EAL: Ask a virtual area of 0x30c00000 bytes EAL: Virtual area found at 0x7fffbda00000 (size = 0x30c00000) EAL: Ask a virtual area of 0x38800000 bytes EAL: Virtual area found at 0x7fff85000000 (size = 0x38800000) EAL: Ask a virtual area of 0x200000 bytes EAL: Virtual area found at 0x7fff84c00000 (size = 0x200000) EAL: Ask a virtual area of 0x296800000 bytes EAL: Virtual area found at 0x7ffcee200000 (size = 0x296800000) EAL: Ask a virtual area of 0x200000 bytes EAL: Virtual area found at 0x7ffcede00000 (size = 0x200000) EAL: Ask a virtual area of 0x200000 bytes EAL: Virtual area found at 0x7ffceda00000 (size = 0x200000) EAL: Ask a virtual area of 0x2ffc00000 bytes EAL: Virtual area found at 0x7ff9edc00000 (size = 0x2ffc00000) EAL: Ask a virtual area of 0x200000 bytes EAL: Virtual area found at 0x7ff9ed800000 (size = 0x200000) EAL: Requesting 6144 pages of size 2MB from socket 0 EAL: Requesting 6144 pages of size 2MB from socket 1 EAL: TSC frequency is ~1895433 KHz EAL: Master core 0 is ready (tid=f7fd0a00) PMD: ENICPMD trace: rte_enic_pmd_init EAL: PCI device 0000:04:00.0 on NUMA socket -1 EAL: probe driver: 8086:1521 rte_igb_pmd EAL: 0000:04:00.0 not managed by UIO driver, skipping EAL: PCI device 0000:04:00.1 on NUMA socket -1 EAL: probe driver: 8086:1521 rte_igb_pmd EAL: 0000:04:00.1 not managed by UIO driver, skipping EAL: PCI device 0000:04:00.2 on NUMA socket -1 EAL: probe driver: 8086:1521 rte_igb_pmd EAL: 0000:04:00.2 not managed by UIO driver, skipping EAL: PCI device 0000:04:00.3 on NUMA socket -1 EAL: probe driver: 8086:1521 rte_igb_pmd EAL: 0000:04:00.3 not managed by UIO driver, skipping EAL: PCI device 0000:84:00.0 on NUMA socket 1 EAL: probe driver: 8086:10fb rte_ixgbe_pmd EAL: PCI memory mapped at 0x7fffee600000 EAL: PCI memory mapped at 0x7fffee680000 PMD: eth_ixgbe_dev_init(): MAC: 2, PHY: 13, SFP+: 5 PMD: eth_ixgbe_dev_init(): port 0 vendorID=0x8086 deviceID=0x10fb EAL: PCI device 0000:84:00.1 on NUMA socket 1 EAL: probe driver: 8086:10fb rte_ixgbe_pmd EAL: PCI memory mapped at 0x7fffee684000 EAL: PCI memory mapped at 0x7fffee704000 PMD: eth_ixgbe_dev_init(): MAC: 2, PHY: 13, SFP+: 6 PMD: eth_ixgbe_dev_init(): port 1 vendorID=0x8086 deviceID=0x10fb [-] Disabling histo engine logs [-] Disabling log_skype_complete [-] Disabling log_chat_complete [-] Disabling log_chat_messages [-] Enabling log_http_complete [-] TCP log level set to 15 (Core + End_to_end + Layer7 + P2P + Options) [-] Video log level set to 31 (Core + End_to_end + Layer7 + VideoInfo + YouTube + Options) (-) Enabling full HTTP URL log [-] Disabling dump engine [Sun May 15 21:11:37 2016] created new outdir tstat_log00/2016_05_15_21_11.out Name:rte_ixgbe_pmd Driver name: rte_ixgbe_pmd Max rx queues: 128 Max tx queues: 128 PCI Adress: 0000:132:00.0 Port 0 Link Down Device supports RSS Device DOES NOT support Flow Director Name:rte_ixgbe_pmd Driver name: rte_ixgbe_pmd Max rx queues: 128 Max tx queues: 128 PCI Adress: 0000:132:00.1 Port 1 Link Up - speed 10000 Mbps - full-duplex Device supports RSS Device DOES NOT support Flow Director Starting instance 2... Executing 'sudo ./build/tstat_dpdk -c 0X0002 -n 4 --proc-type=secondary -- -m 2 -p 1 &' EAL: Detected lcore 0 as core 0 on socket 0 EAL: Detected lcore 1 as core 1 on socket 0 EAL: Detected lcore 2 as core 2 on socket 0 EAL: Detected lcore 3 as core 3 on socket 0 EAL: Detected lcore 4 as core 4 on socket 0 EAL: Detected lcore 5 as core 5 on socket 0 EAL: Detected lcore 6 as core 0 on socket 1 EAL: Detected lcore 7 as core 1 on socket 1 EAL: Detected lcore 8 as core 2 on socket 1 EAL: Detected lcore 9 as core 3 on socket 1 EAL: Detected lcore 10 as core 4 on socket 1 EAL: Detected lcore 11 as core 5 on socket 1 EAL: Detected lcore 12 as core 0 on socket 0 EAL: Detected lcore 13 as core 1 on socket 0 EAL: Detected lcore 14 as core 2 on socket 0 EAL: Detected lcore 15 as core 3 on socket 0 EAL: Detected lcore 16 as core 4 on socket 0 EAL: Detected lcore 17 as core 5 on socket 0 EAL: Detected lcore 18 as core 0 on socket 1 EAL: Detected lcore 19 as core 1 on socket 1 EAL: Detected lcore 20 as core 2 on socket 1 EAL: Detected lcore 21 as core 3 on socket 1 EAL: Detected lcore 22 as core 4 on socket 1 EAL: Detected lcore 23 as core 5 on socket 1 EAL: Support maximum 128 logical core(s) by configuration. EAL: Detected 24 lcore(s) sh: 1: cannot create /dev/cpuset/tstat-dpdk00/cpuset.cpus: Permission denied EAL: cannot connect to primary process! EAL: VFIO support could not be initialized EAL: Setting up memory... EAL: Analysing 12288 files EAL: Mapped segment 0 of size 0x30c00000 EAL: Mapped segment 1 of size 0x38800000 EAL: Mapped segment 2 of size 0x200000 sh: 1: cannot create /dev/cpuset/tstat-dpdk00/cpuset.mems: Permission denied sh: 1: cannot create /dev/cpuset/cpuset.sched_load_balance: Permission denied sh: 1: cannot create /dev/cpuset/cpuset.cpu_exclusive: Permission denied EAL: Mapped segment 3 of size 0x296800000 EAL: Mapped segment 4 of size 0x200000 EAL: Mapped segment 5 of size 0x200000 /bin/echo: write error: No space left on device NIC polling thread created with tid:2691 EAL: Mapped segment 6 of size 0x2ffc00000 EAL: Mapped segment 7 of size 0x200000 EAL: memzone_reserve_aligned_thread_unsafe(): memzone already exists RING: Cannot reserve memory EAL: TSC frequency is ~1895433 KHz EAL: Master core 1 is ready (tid=f7fd0a00) PMD: ENICPMD trace: rte_enic_pmd_init EAL: PCI device 0000:04:00.0 on NUMA socket -1 EAL: probe driver: 8086:1521 rte_igb_pmd EAL: Cannot find resource for device EAL: PCI device 0000:04:00.1 on NUMA socket -1 EAL: probe driver: 8086:1521 rte_igb_pmd EAL: Cannot find resource for device EAL: PCI device 0000:04:00.2 on NUMA socket -1 EAL: probe driver: 8086:1521 rte_igb_pmd EAL: Cannot find resource for device EAL: PCI device 0000:04:00.3 on NUMA socket -1 EAL: probe driver: 8086:1521 rte_igb_pmd EAL: Cannot find resource for device EAL: PCI device 0000:84:00.0 on NUMA socket 1 EAL: probe driver: 8086:10fb rte_ixgbe_pmd EAL: PCI memory mapped at 0x7fffee600000 EAL: PCI memory mapped at 0x7fffee680000 PMD: set_tx_function(): Using simple tx code path PMD: set_tx_function(): Vector tx enabled. EAL: PCI device 0000:84:00.1 on NUMA socket 1 EAL: probe driver: 8086:10fb rte_ixgbe_pmd EAL: PCI memory mapped at 0x7fffee684000 EAL: PCI memory mapped at 0x7fffee704000 PMD: set_tx_function(): Using simple tx code path PMD: set_tx_function(): Vector tx enabled. [-] Disabling histo engine logs [-] Disabling log_skype_complete [-] Disabling log_chat_complete [-] Disabling log_chat_messages [-] Enabling log_http_complete [-] TCP log level set to 15 (Core + End_to_end + Layer7 + P2P + Options) [-] Video log level set to 31 (Core + End_to_end + Layer7 + VideoInfo + YouTube + Options) (-) Enabling full HTTP URL log [-] Disabling dump engine [Sun May 15 21:11:39 2016] created new outdir tstat_log01/2016_05_15_21_11.out All the 2 instances are up. mount: cpuset already mounted or /dev/cpuset busy mount: according to mtab, cpuset is already mounted on /dev/cpuset sh: 1: cannot create /dev/cpuset/tstat-dpdk01/cpuset.cpus: Permission denied sh: 1: cannot create /dev/cpuset/tstat-dpdk01/cpuset.mems: Permission denied sh: 1: cannot create /dev/cpuset/cpuset.sched_load_balance: Permission denied sh: 1: cannot create /dev/cpuset/cpuset.cpu_exclusive: Permission denied /bin/echo: write error: No space left on device NIC polling thread created with tid:2709 Instance: 0 Pkts: 290704 Avg: 0.598us Max: 109.379us StdDev: 0.436 TCP cl.: 0 UDP cl.: 0 Rate: 0.295Mpps Loss: 2.123Mpps Mem. occupation: 2% ( 3% in tot.) <--------L-O-S-I-N-G--------- Instance: 1 Pkts: 1004140 Avg: 0.600us Max: 77.247us StdDev: 0.379 TCP cl.: 0 UDP cl.: 0 Rate: 1.005Mpps Loss: 7.457Mpps Mem. occupation: 1% ( 3% in tot.) <--------L-O-S-I-N-G---------

Instance: 0 Pkts: 988389 Avg: 0.596us Max: 429.116us StdDev: 0.568 TCP cl.: 0 UDP cl.: 0 Rate: 0.987Mpps Loss: 7.462Mpps Mem. occupation: 1% ( 3% in tot.) <--------L-O-S-I-N-G--------- Instance: 1 Pkts: 931732 Avg: 0.599us Max: 156.110us StdDev: 0.412 TCP cl.: 0 UDP cl.: 0 Rate: 0.931Mpps Loss: 7.516Mpps Mem. occupation: 1% ( 3% in tot.) <--------L-O-S-I-N-G---------

Instance: 0 Pkts: 931729 Avg: 0.598us Max: 48.618us StdDev: 0.386 TCP cl.: 0 UDP cl.: 0 Rate: 0.930Mpps Loss: 7.517Mpps Mem. occupation: 0% ( 3% in tot.) <--------L-O-S-I-N-G--------- Instance: 1 Pkts: 925103 Avg: 0.607us Max: 8876.119us StdDev: 0.384 TCP cl.: 0 UDP cl.: 0 Rate: 0.927Mpps Loss: 7.519Mpps Mem. occupation: 2% ( 4% in tot.) <--------L-O-S-I-N-G---------

Instance: 0 Pkts: 916799 Avg: 0.595us Max: 125.181us StdDev: 0.406 TCP cl.: 0 UDP cl.: 0 Rate: 0.910Mpps Loss: 7.533Mpps Mem. occupation: 1% ( 3% in tot.) <--------L-O-S-I-N-G--------- Instance: 1 Pkts: 974367 Avg: 0.595us Max: 53.016us StdDev: 0.361 TCP cl.: 0 UDP cl.: 0 Rate: 0.972Mpps Loss: 7.474Mpps Mem. occupation: 1% ( 3% in tot.) <--------L-O-S-I-N-G---------

Instance: 0 Pkts: 1004911 Avg: 0.592us Max: 123.558us StdDev: 0.367 TCP cl.: 0 UDP cl.: 0 Rate: 1.000Mpps Loss: 7.444Mpps Mem. occupation: 1% ( 2% in tot.) <--------L-O-S-I-N-G--------- Instance: 1 Pkts: 1022199 Avg: 0.599us Max: 351.818us StdDev: 0.521 TCP cl.: 0 UDP cl.: 0 Rate: 1.020Mpps Loss: 7.430Mpps Mem. occupation: 0% ( 2% in tot.) <--------L-O-S-I-N-G---------

Instance: 0 Pkts: 1005877 Avg: 0.599us Max: 121.927us StdDev: 0.405 TCP cl.: 0 UDP cl.: 0 Rate: 1.011Mpps Loss: 7.437Mpps Mem. occupation: 1% ( 3% in tot.) <--------L-O-S-I-N-G--------- Instance: 1 Pkts: 952083 Avg: 0.599us Max: 120.399us StdDev: 0.413 TCP cl.: 0 UDP cl.: 0 Rate: 0.946Mpps Loss: 7.497Mpps Mem. occupation: 1% ( 2% in tot.) <--------L-O-S-I-N-G---------

Instance: 0 Pkts: 930268 Avg: 0.598us Max: 469.419us StdDev: 0.704 TCP cl.: 0 UDP cl.: 0 Rate: 0.930Mpps Loss: 7.517Mpps Mem. occupation: 1% ( 3% in tot.) <--------L-O-S-I-N-G--------- ^C PORT: 0 Rx: 6358771 Drp: 49276458 Tot: 55635229 Perc: 88.571% Queue 0 Rx: 3180338 Drp: 0 Tot: 3180338 Perc: 0.000% Queue 1 Rx: 3178433 Drp: 0 Tot: 3178433 Perc: 0.000% PORT: 1 Rx: 6337776 Drp: 49298466 Tot: 55636242 Perc: 88.609% Queue 0 Rx: 3167343 Drp: 0 Tot: 3167343 Perc: 0.000% Queue 1 Rx: 3170433 Drp: 0 Tot: 3170433 Perc: 0.000%

ISTANCE: 0 Rx: 6344269 Drp: 0 Tot: 6344269 Perc: 0.000% TSTAT STATISTICS: Total packets: 6344269 TCP flows: 0 UDP flows: 0 Time elapsed: 8.704s

ISTANCE: 1 Rx: 6351493 Drp: 0 Tot: 6351493 Perc: 0.000% TSTAT STATISTICS: Total packets: 6351493 TCP flows: 0 UDP flows: 0 Time elapsed: 7.025s root@HaoChuan:/home/xjtu/tstat-dpdk#

marty90 commented 8 years ago

I think you are overloading the cores. Trying increasing the number of cores / instances used. You should not exceed 1-1.5 Mpps for each core. Let me know the performance you achieve. P.S. Having some information about the traffic you are analysing might help me to give better hints. Is real or synthetic?