Cannot send packets

[laochonlam]

Hi, thanks for your open-source in the dpdk solution. I tried this and I am not able to send packets.

When I use this DPDK, every installation instruction works perfectly until the last step - send packet via pktgen. It should send packets, but I can't receive any packets on the other side of the NIC. Do you have any better way to debug it or how you debug it? Thanks!

Here is the log pktgen generated.

$ ~/git/open-nic-dpdk_workspace/pktgen-dpdk-pktgen-20.11.3$ sudo usr/local/bin/pktgen -a 5e:00.0 -d librte_net_qdma.so -l 0-2 -n 2 -a 5d:00.0  -- -m [6:7].0

Copyright (c) <2010-2020>, Intel Corporation. All rights reserved. Powered by DPDK
EAL: Detected 20 lcore(s)
EAL: Detected 2 NUMA nodes
EAL: Multi-process socket /var/run/dpdk/rte/mp_socket
EAL: Selected IOVA mode 'VA'
EAL: 16 hugepages of size 1073741824 reserved, but no mounted hugetlbfs found for that size
EAL: Probing VFIO support...
EAL: VFIO support initialized
EAL:   using IOMMU type 1 (Type 1)
EAL: Probe PCI driver: net_qdma (10ee:903f) device: 0000:5e:00.0 (socket 0)
Device Type: Soft IP
IP Type: EQDMA Soft IP
Vivado Release: vivado 2020.2
PMD: qdma_get_hw_version(): QDMA RTL VERSION : RTL Base

PMD: qdma_get_hw_version(): QDMA DEVICE TYPE : Soft IP

PMD: qdma_get_hw_version(): QDMA VIVADO RELEASE ID : vivado 2020.2

PMD: qdma_identify_bars(): QDMA config bar idx :0

PMD: qdma_identify_bars(): QDMA AXI Master Lite bar idx :2

PMD: qdma_identify_bars(): QDMA AXI Bridge Master bar idx :-1

PMD: qdma_eth_dev_init(): QDMA device driver probe:
PMD: qdma_device_attributes_get(): qmax = 512, mm 1, st 1.

PMD: qdma_eth_dev_init(): PCI max bus number : 0x5e
PMD: qdma_eth_dev_init(): PF function ID: 0
PMD: QDMA PMD VERSION: 2020.2.1
qdma_dev_entry_create: Created the dev entry successfully
EAL: No legacy callbacks, legacy socket not created

*** Copyright (c) <2010-2020>, Intel Corporation. All rights reserved.
*** Pktgen  created by: Keith Wiles -- >>> Powered by DPDK <<<

 Port: Name         IfIndex Alias        NUMA  PCI
    0: net_qdma        0                   0   10ee:903f/5e:00.0

Initialize Port 0 -- TxQ 1, RxQ 1
PMD: qdma_dev_configure(): Configure the qdma engines

PMD: qdma_dev_configure(): Bus: 0x0, PF-0(DEVFN) queue_base: 0

PMD: qdma_dev_rx_queue_setup(): Configuring Rx queue id:0

PMD: qdma_dev_tx_queue_setup(): Configuring Tx queue id:0 with 512 desc

PMD: qdma_dev_tx_queue_setup(): Tx ring phys addr: 0x10AA99000, Tx Ring virt addr: 0x10AA99000
Src MAC 15:16:17:18:19:1a

PMD: qdma_dev_start(): qdma-dev-start: Starting

PMD: qdma_dev_link_update(): Link update done

WARNING: Nothing to do on lcore 1: exiting
WARNING: Nothing to do on lcore 2: exiting
- Ports 0-0 of 1   <Main Page>  Copyright (c) <2010-2020>, Intel Corporation
  Flags:Port        : -------Single      :0PMD: qdma_dev_link_update(): Link update done
|ink State          :        <UP-100000-FD>      ---Total Rate---
Pkts/s Max/Rx       : -------Single      :0PMD: qdma_dev_link_update(): Link update done
       Max/Tx       :        <UP-100000-FD>                   0/0
MBits/s Rx/Tx       :                   0/0                   0/0
Broadcast           :                   0/0                   0/0
Multicast           :                   0/0                   0/0
Sizes 64            :                     0
      65-127        :                     0
      128-255       :                     0
      256-511       :                     0
      512-1023      :                     0
      1024-1518     :                     0
Runts/Jumbos        :                     0
ARP/ICMP Pkts       :                     0
Errors Rx/Tx        :                   0/0
Total Rx Pkts       :                   0/0
      Tx Pkts       :                   0/0
      Rx MBs        :                     0
      Tx MBs        :                     0
                    :                     0
Pattern Type        :                     0
Tx Count/% Rate     :         Forever /100%
Pkt Size/Tx Burst   :             64 /   32
TTL/Port Src/Dest   :         4/ 1234/ 5678
Pkt Type:VLAN ID    :       IPv4 / TCP:0001
802.1p CoS/DSCP/IPP :             0/  0/  0
VxLAN Flg/Grp/vid   :      0000/    0/    0
IP  Destination     :           192.168.1.1
    Source          :        192.168.0.1/24
MAC Destination     :     00:00:00:00:00:00
    Source          :     15:16:17:18:19:1a
PCI Vendor/Addr     :     10ee:903f/5e:00.0

-- Pktgen 20.11.3 (DPDK 20.11.0)  Powered by DPDK  (pid:7051) -----------------
** Version: DPDK 20.11.0, Command Line Interface without timers
Pktgen:/>

---

I use U250 and tested with open-nic-driver.

Lam

[JinghanHuang]

Hi Lam,

I have a similar question. Could you share how you solved this problem?

I use U280. Thanks in advance!

Best, Jinghan

[laochonlam]

No I am not able to solve this problem.

Lm

[cneely-amd]

Hi, I suggest checking what the CMAC RX link status along with some of the CMAC registers (port 0: 0x8204 or port 1: 0xC204). The RX link status need to be 0x3 for transmitting packets. Examples of link status errors are reading back values of 0xC0 or 0xE0.

[cneely-amd]

Another possible cause might be if the OpenNIC is connected to a switch with RS-FEC enabled on that port. The current steps in the instructions didn't cover the step for writing the CMAC's register for enabling RS-FEC and followed by writing to SYSCFG_OFFSET_SHELL_RESET to reset the CMAC subsystem related logic, which the Linux kernel driver performs in lines 190-198 of "onic_hardware.c". There was a bug fix checked in today to open-nic-driver related to the issue of resets after enabling RS-FEC.
https://github.com/Xilinx/open-nic-driver/commit/59af3287b19c7ed35888f49a5cc3ede03d84f11a

[function47]

Hi, I suggest checking what the CMAC RX link status along with some of the CMAC registers (port 0: 0x8204 or port 1: 0xC204). The RX link status need to be 0x3 for transmitting packets. Examples of link status errors are reading back values of 0xC0 or 0xE0.

Hi, all,

I can send packets using pktgen (although I can send packets, the statistics is quite off). Each time I quit pktgen and unbind the dpdk, I found the CMAC link status (0x8204) became 0xC0. Is there any solution to this issue?

Thanks!