Nolink when porting to ADM-PCIE-9V3

[liubenyuan]

Hi I tried to build a simple lookback example on ADM-PCIE-9V3, where only CMAC and network-stack-core are kept, the PCIE part are removed. The block diagram is as follows:

The mm2s and s2mm is connected in a loopback mode. We aim to first to test that a Mellanox MCX623106A 100Gbe card can have a link to ADM-PCIE-9V3.

The GTY configurations of ADM-PCIE-9V3 is

} elseif {${fpga_part} eq "xcvu3p-ffvc1517-2-e"} {
  set gt_ref_clk 161.1328125
  # DIFF 300MHz, MMCM 125 MHz for free running clock
  set freerunningclock 125
  switch ${interface} {
    "1" {
      # QSFP1, QUAD 127
      set core_selection  CMACE4_X0Y1
      set group_selection X0Y12~X0Y15
      set interface_number 1
    }
    default {
      # QSFP0, QUAD 128
      set core_selection  CMACE4_X0Y2
      set group_selection X0Y16~X0Y19
      set interface_number 0
    }
  }
} else {
  puts "unknown part"
  return -1
}

The is a freerunning clock, 125MHz, which is generated from onboard 300MHz clock via a MMCM. This MMCM also output an 250MHz clock (called cclk), which is for user modules, and in this simple example, cclk is connect to ap_clk of 100G-network-stack-core and CMAC core.

Note that the AXI4Lite interface of both 100G-network-stack-core and CMAC are not connected.

After I write this bitstream to ADM-PCIE-9V3, and connect an 100G-SR4 cable from ADM-PCIE-9V3 to Mellanox 623106A, the card shows no-link.

The configuration of mellanox card is as follows, where AN is diabled, 100G_4X is made default and RS-FEC is also disabled,

Operational Info
----------------
State                              : Polling
Physical state                     : ETH_AN_FSM_ENABLE
Speed                              : N/A
Width                              : N/A
FEC                                : N/A
Loopback Mode                      : No Loopback
Auto Negotiation                   : FORCE - 100G_4X

Supported Info
--------------
Enabled Link Speed (Ext.)          : 0x00000200 (100G_4X)
Supported Cable Speed (Ext.)       : 0x000002f2 (100G_4X,50G_2X,40G,25G,10G,1G)

Troubleshooting Info
--------------------
Status Opcode                      : 36
Group Opcode                       : PHY FW
Recommendation                     : Other issues

Tool Information
----------------
Firmware Version                   : 22.36.1010
amBER Version                      : 2.17
MFT Version                        : mft 4.25.0-62

Module Info
-----------
Identifier                         : QSFP28
Compliance                         : 100G AOC (Active Optical Cable) or 25GAUI C2M AOC with FEC
Cable Technology                   : 850 nm VCSEL
Cable Type                         : Active cable (active copper / optics)
OUI                                : Other
Vendor Name                        : LUXSHARE-ICT
Vendor Part Number                 : PA0QQ1112-SD-R
Vendor Serial Number               : A29WA005
Rev                                : B
Wavelength [nm]                    : 850
Transfer Distance [m]              : 1
Attenuation (5g,7g,12g) [dB]       : N/A
FW Version                         : N/A
Digital Diagnostic Monitoring      : Yes
Power Class                        : 2.0 W max
CDR RX                             : ON,ON,ON,ON
CDR TX                             : ON,ON,ON,ON
LOS Alarm                          : N/A
Temperature [C]                    : 39 [-5..80]
Voltage [mV]                       : 3259 [2900..3600]
Bias Current [mA]                  : 6.360,16.424,6.490,6.538 [0.5..20]
Rx Power Current [dBm]             : 0,0,0,0 [-12..4]
Tx Power Current [dBm]             : -2,2,-2,-2 [-12..4]
SNR Media Lanes [dB]               : N/A
SNR Host Lanes [dB]                : N/A
IB Cable Width                     : 1x,2x,4x
Memory Map Revision                : 8
Linear Direct Drive                : 0
Cable Breakout                     : Channels implemented [1,2,3,4]/Far end is unspecified
SMF Length                         : N/A
MAX Power                          : 0
Cable Rx AMP                       : 0
Cable Rx Emphasis                  : 0
Cable Rx Post Emphasis             : 0
Cable Tx Equalization              : 0
Wavelength Tolerance               : 10.0nm
Module State                       : N/A
DataPath state [per lane]          : N/A,N/A,N/A,N/A
Rx Output Valid [per lane]         : 0,0,0,0
Nominal bit rate                   : 25.750Gb/s
Rx Power Type                      : Average power
Manufacturing Date                 : 30_09_22
Active Set Host Compliance Code    : N/A
Active Set Media Compliance Code   : N/A
Error Code Response                : N/A
Module FW Fault                    : N/A
DataPath FW Fault                  : N/A
Tx Fault [per lane]                : 0,0,0,0
Tx LOS [per lane]                  : 0,0,0,0
Tx CDR LOL [per lane]              : 0,0,0,0
Rx LOS [per lane]                  : 0,0,0,0
Rx CDR LOL [per lane]              : 0,0,0,0
Tx Adaptive EQ Fault [per lane]    : 0,0,0,0

The IP address are set to 192.168.0.9, where the FPGA IP address is 192.168.0.5.

Any hint on this problem? Thanks!

Build Issues

$ vitis --version

****** Xilinx Vitis Development Environment
****** Vitis v2022.2 (64-bit)
  **** SW Build 3671529 on 2022-10-13-17:52:08
    ** Copyright 1986-2022 Xilinx, Inc. All Rights Reserved.

$ uname -a
Linux archlinux 6.5.3-native_amd-xanmod1-1 #1 SMP PREEMPT_DYNAMIC Thu, 14 Sep 2023 10:21:49 +0000 x86_64 GNU/Linux

[mariodruiz]

Hi @liubenyuan,

This board is not supported in this project. I suggest you seek support on the Xilinx forums.

[liubenyuan]

Hi @liubenyuan,

This board is not supported in this project. I suggest you seek support on the Xilinx forums.

Thank you. One more question, is it ok to build a RTL only bitstream .bit like figure 1? Without configuration to the AXI4Lite registers of 100G-network-core and the CMAC?

[mariodruiz]

One more question, is it ok to build a RTL only bitstream .bit like figure 1? Without configuration to the AXI4Lite registers of 100G-network-core and the CMAC?

Technically yes, but it is recommended to have the AXI4-Lite interface for debug and configuration.