I have test dpdk2disk with default version dpdk inlude this. but get error message like this.Pleas help this issue. root@knot-onesys:/home/dhb/DPDK2disk# ./scripts/capture0.sh /home/dhb/ EAL: Detected 8 lcore(s) EAL: Probing VFIO support... EAL: PCI device 0000:02:00.0 on NUMA socket -1 EAL: Invalid NUMA socket, default to 0 EAL: probe driver: 8086:100f net_e1000_em EAL: PCI device 0000:02:01.0 on NUMA socket -1 EAL: Invalid NUMA socket, default to 0 EAL: probe driver: 8086:100f net_e1000_em EAL: PCI device 0000:02:02.0 on NUMA socket -1 EAL: Invalid NUMA socket, default to 0 EAL: probe driver: 8086:100f net_e1000_em EAL: PCI device 0000:02:03.0 on NUMA socket -1 EAL: Invalid NUMA socket, default to 0 EAL: probe driver: 8086:100f net_e1000_em EAL: PCI device 0000:03:00.0 on NUMA socket -1 EAL: Invalid NUMA socket, default to 0 EAL: probe driver: 8086:10fb net_ixgbe [EXTERNAL] Se define un WSlave en el Core 4 [EXTERNAL] Carpeta establecida a /home/dhb/. [EXTERNAL] Tamaño maximo de fichero establecido a 4294967296 bytes Creating the mbuf pool for socket 0 ... Creating ring to connect I/O lcore 1 (socket 0) with worker lcore 3 ... Creating ring to connect worker lcore 3 with TX port 0 (through I/O lcore 2) (socket 0) ... Initializing NIC port 0 ... Initializing NIC port 0 RX queue 0 ... Initializing NIC port 0 TX queue 0 ... PMD: ixgbe_dev_link_status_print(): Port 0: Link Down

Checking link status.................................done Port 0 Link Up - speed 1000 Mbps - full-duplex [EXTERNAL] Reservando recursos...Malloc - ERROR! /home/dhb/DPDK2disk root@knot-onesys:/home/dhb/DPDK2disk#

Hi, First of all, sorry for that "in spanish" error message. The problem is that this app uses LOTS of ram, see https://github.com/hpcn-uam/DPDK2disk/blob/master/src/external.c at line 162.

It is trying to allocate sizeof(BUFFERKIND)*diskBufferSize witch means 1*1024*1024*1024 bytes (1GiB) 4 times (4GiB in total, sometimes caused by dpdk overheads it can be extended up to 8GiB) Show me your hugepage setup.

Also I do not recommend to use /home as output directory, you probably want to use some kind of raid 0 or NVMe array, specially if you want to capture >5Gbit or so.

Hi ralequi: That is very kind of you to help me , as I was new to test DPDK, and my test lab use Vmware Esxi 6.5 run ubuntu 16.04 with Vxnet3 and two intel X520(pci passthrougth).I have other test server with SSD installed for more fast capture ability, but not ready by now. so this is my first test.I alway get DPDK errror with VM. and I may not repeat this issue. Here is steps I try: 1.check DPDK version(seems the same with DPDK2disk git) root@knot-onesys:/home/dhb/DPDK2disk/dpdk# pwd /home/dhb/DPDK2disk/dpdk root@knot-onesys:/home/dhb/DPDK2disk/dpdk# git branch

(HEAD detached at d84aad6) master root@knot-onesys:/home/dhb/DPDK2disk/dpdk# 2.try set up DPDK ./usertools/dpdk-setup.sh Configuration done using x86_64-native-linuxapp-gcc == Build lib == Build lib/librte_compat == Build lib/librte_eal == Build lib/librte_eal/common == Build lib/librte_eal/linuxapp == Build lib/librte_eal/linuxapp/eal == Build lib/librte_eal/linuxapp/igb_uio ........
1. set hugepages Option: 21

Removing currently reserved hugepages Unmounting /mnt/huge and removing directory

Input the number of 2048kB hugepages for each node Example: to have 128MB of hugepages available per node in a 2MB huge page system, enter '64' to reserve 64 * 2MB pages on each node Number of pages for node0: 2048 Reserving hugepages Creating /mnt/huge and mounting as hugetlbfs

Press enter to continue ... 4.check hugepages

cat /proc/meminfo | grep Huge

AnonHugePages: 8192 kB HugePages_Total: 2048 HugePages_Free: 2048 HugePages_Rsvd: 0 HugePages_Surp: 0 Hugepagesize: 2048 kB 5.check interface

dpdk_nic_bind --status

Network devices using DPDK-compatible driver

0000:03:00.0 '82599ES 10-Gigabit SFI/SFP+ Network Connection' drv=igb_uio unused=ixgbe 0000:0b:00.0 '82599ES 10-Gigabit SFI/SFP+ Network Connection' drv=igb_uio unused=ixgbe 6.try test capture

./scripts/capture0.sh /home/dhb/

I get error message : EAL: Detected 8 lcore(s) EAL: lcore 8 unavailable EAL: invalid coremask

Here is my cpu layout: ./dpdk/usertools/cpu_layout.py

Core and Socket Information (as reported by '/sys/devices/system/cpu')

cores = [0, 1, 2, 3, 4, 5, 6, 7] sockets = [0]

   Socket 0   
   --------

Core 0 [0]
Core 1 [1]
Core 2 [2]
Core 3 [3]
Core 4 [4]
Core 5 [5]
Core 6 [6]
Core 7 [7]
I have "vi ./scripts/capture0.sh" change "sudo build/app/hpcn_n2d -c FFF" to "sudo build/app/hpcn_n2d -c 0xff" (I am not sure if this is right?)

try ./scripts/capture0.sh /home/dhb/

get error messge:

EAL: Detected 8 lcore(s) EAL: Probing VFIO support... EAL: PCI device 0000:02:00.0 on NUMA socket -1 EAL: Invalid NUMA socket, default to 0 EAL: probe driver: 8086:100f net_e1000_em EAL: PCI device 0000:03:00.0 on NUMA socket -1 EAL: Invalid NUMA socket, default to 0 EAL: probe driver: 8086:10fb net_ixgbe EAL: PCI device 0000:0b:00.0 on NUMA socket -1 EAL: Invalid NUMA socket, default to 0 EAL: probe driver: 8086:10fb net_ixgbe EAL: PCI device 0000:13:00.0 on NUMA socket -1 EAL: Invalid NUMA socket, default to 0 EAL: probe driver: 15ad:7b0 net_vmxnet3 EAL: PCI device 0000:1b:00.0 on NUMA socket -1 EAL: Invalid NUMA socket, default to 0 EAL: probe driver: 15ad:7b0 net_vmxnet3 [EXTERNAL] Se define un WSlave en el Core 4 [EXTERNAL] Carpeta establecida a /home/dhb/. [EXTERNAL] Tamaño maximo de fichero establecido a 4294967296 bytes Creating the mbuf pool for socket 0 ... Creating ring to connect I/O lcore 1 (socket 0) with worker lcore 3 ... Creating ring to connect worker lcore 3 with TX port 0 (through I/O lcore 2) (socket 0) ... Initializing NIC port 0 ... Initializing NIC port 0 RX queue 0 ... Initializing NIC port 0 TX queue 0 ... PMD: ixgbe_dev_link_status_print(): Port 0: Link Down

Checking link status..........................done Port 0 Link Up - speed 1000 Mbps - full-duplex [EXTERNAL] Reservando recursos...Malloc - ERROR! /home/dhb/DPDK2disk

Thank you for your help.

Hi, Sorry to take too long on response. I absolutely forget about it :-(. (too much work to remember everything u probably know...)

First of all. VMs have some degradation on DPDK. It's not like a terrible thing but it is present. Read the results present here: https://www.dit.upm.es/~jlopez/publicaciones/comnet19.pdf
Second, SSD can be nice for some initial tests, but a single ssd can't handle network capture. If I remember correctly, you need at least 4 SSD to capture at 10gbit, but even with that, it is not posible to keep it capturing for long time, since you would have SSD degradation, witch it's only fix is to perform a "secure-erase". Keep that in mind when using SSDs.
Third, DPDK recomends 1G hugepages. Insert this options to your kernel at boot: hugepagesz=1G default_hugepagesz=1G hugepages=16. You should put that on host and VMs. Also, vmware should be aware and force to allocate VM memory over hugepages to get the best performance with any DPDK app.
4º, the scripts stored at /scripts, are ad-hoc. With that said, open it and change whatever you want, in this case, the coremask (-c option to the underlay application). In this case -c 0xff (As you have 8 cores) may be an option. Remember that the smallest core in the mask is going to be used by dpdk internally. Answering to your "7º", that mask is correct for your setup.
5º, I'm not 100% sure, but the memory problem may come from using 2MB pages and low number of pages. You have allocated 4GB, (which in fact is exactly the memory required for that malloc, you need some extra mem for mbufs, etc). Also the rte_malloc functions requires to have the memory "physically contiguous" (not virtual), a fact that is not "as easy as you may think" to achieve with 2MB pages, specially if the host is running for more than a couple of hours or have some heavy mem processes (such a graphic interface).

And to conclude.... If you are going to use this in an academic work, I'm open to work with you in deep with the simplest condition of being a co-author ;-)

hpcn-uam / DPDK2disk

[EXTERNAL] Reservando recursos...Malloc - ERROR! Please help #1