search

liuw666-bruce / DPDK-with-Corundum

A DPDK repo with Corundum driver
25 stars 13 forks source link

Question: Compatible version of Corundum #2

Open nightseas opened 1 year ago

nightseas commented 1 year ago

Hi,

Just would like to know which version/commit of Corundum was tested with this repo?

Thanks & BR.

liuw666-bruce commented 1 year ago

I downloaded the project of corundum at 2020.11.2

nightseas commented 1 year ago

Hi,

The 2020.11 version of Corundum is actually an AXI lib instead of full functional NIC.

Updated, made a mistake on the commit HEAD. I'll try earlier version of FPGA and see if ti's still the same problem.

I tried with HEAD commit 38f7666, which can be successfully initialized. But Corundum only had TX traffic and no packet was received during pktgen test.

Any idea why the RX didn't work? Kernel message show a question on interrupt. Does it mean that there's incorrect IRQ configuration on corundum?

$ dmesg
...
[ 2086.790533] uio_pci_generic 0000:17:00.0: No IRQ assigned to device: no support for interrupts?
[ 2106.338614] ice 0000:ca:00.0: Removed PTP clock

pktgen command:

sudo ./build/app/pktgen -v -c 0xFFF --proc-type auto --log-level=8 -- -P -v -m "[4/6:8/10].0,[5/7:9/11].1"

Hardware setup is CPU socket 0 <-> Port 0: Corundum (ADM-PCIe-9V3) <-> 100G DAC <-> Port 1: Intel E810 <-> CPU socket 1. image

Thanks & BR.

nightseas commented 1 year ago

Same result with 2022 Oct version.

commit 53f4275ea2325df81d481782659b42dff0f9a3bd (HEAD)
Author: Alex Forencich <alex@alexforencich.com>
Date:   Tue Oct 13 23:52:52 2020 -0700

    Add output registers for I2C interface to improve timing
liuw666-bruce commented 1 year ago

Did you try ARP protocol? The board works well with original Linux driver, right? And I don't think it is related to interrupt, this DPDK driver doesn't support interrupt.

nightseas commented 1 year ago

Did you try ARP protocol? The board works well with original Linux driver, right? And I don't think it is related to interrupt, this DPDK driver doesn't support interrupt.

Yes, only ARP can be received. And with kernel driver the board works fine.

nightseas commented 1 year ago

It's an issue of improper handling RSS in the driver. The RSS mask shall be dynamically configured based on how many cores/queues are used for RX.

zqchinaone commented 1 year ago

@nightseas which version/commit of Corundum was tested with this repo? thanks

nightseas commented 1 year ago

38f7666 is the commit that I've been testing. It's just a fork from Corundum so that you will find the same commit on upstream repo.

I'm planning to port it to a later version of Corundum (before QP is introduced), and perhaps DPDK too.

zqchinaone commented 1 year ago

好的，多谢。

liuw666-bruce commented 1 year ago

It's an issue of improper handling RSS in the driver. The RSS mask shall be dynamically configured based on how many cores/queues are used for RX.

I think you are right, I set the rx_queue_count to 16 because the driver failed to alloc memory (too much) when the rx_queue_count is 8192. I didn't have testing devices to fix the issue now. Anybody who need to temporarily pass it can try to set the (especially for rx and rx complete)queue number to 16 in HW.

define MQNIC_MAX_EVENT_RINGS 2//256

define MQNIC_MAX_TX_RINGS 16//8192

define MQNIC_MAX_TX_CPL_RINGS 16//8192

define MQNIC_MAX_RX_RINGS 16// 8192

define MQNIC_MAX_RX_CPL_RINGS 16//8192

liuw666-bruce commented 1 year ago

It's an issue of improper handling RSS in the driver. The RSS mask shall be dynamically configured based on how many cores/queues are used for RX.

I think you are right, I set the rx_queue_count to 16 because the driver failed to alloc memory (too much) when the rx_queue_count is 8192. I didn't have testing devices to fix the issue now. Anybody who need to temporarily pass it can try to set the (especially for rx and rx complete)queue number to 16 in HW.

define MQNIC_MAX_EVENT_RINGS 2//256 #define MQNIC_MAX_TX_RINGS 16//8192 #define MQNIC_MAX_TX_CPL_RINGS 16//8192 #define MQNIC_MAX_RX_RINGS 16// 8192 #define MQNIC_MAX_RX_CPL_RINGS 16//8192

It is also OK to open more queues in HW, but don't forget to modify this MICROs in SW at the same time.