[NiteFury] Device not enumerating with x1 x2

[luigifcruz]

Hi, I successfully tested the sqrl_acorn.py example with the default pcie_x4 setting. Later, I edited the platform file and added x1 and x2 configurations. I regenerated the gateware after modifying the target configuration to pcie_x2. I also changed the bus_width to 64 because Vivado complained with 128. After flashing and rebooting the machine, the FPGA was not enumerating anymore. I checked the generated files and everything looks in order. The same happens with x1.

[luigifcruz]

Upon further inspection, the pins from the constraints files are correct. But after synthesizing and checking the IO Report the pins are different. It looks like Vivado somehow ignores the constraints and changes the assignment.

# pcie_x2:0.clk_p
set_property LOC F6 [get_ports {pcie_x2_clk_p}]

# pcie_x2:0.clk_n
set_property LOC E6 [get_ports {pcie_x2_clk_n}]

# pcie_x2:0.rx_p
set_property LOC B10 [get_ports {pcie_x2_rx_p[0]}]

# pcie_x2:0.rx_p
set_property LOC B8 [get_ports {pcie_x2_rx_p[1]}]

# pcie_x2:0.rx_n
set_property LOC A10 [get_ports {pcie_x2_rx_n[0]}]

# pcie_x2:0.rx_n
set_property LOC A8 [get_ports {pcie_x2_rx_n[1]}]

# pcie_x2:0.tx_p
set_property LOC B6 [get_ports {pcie_x2_tx_p[0]}]

# pcie_x2:0.tx_p
set_property LOC B4 [get_ports {pcie_x2_tx_p[1]}]

# pcie_x2:0.tx_n
set_property LOC A6 [get_ports {pcie_x2_tx_n[0]}]

# pcie_x2:0.tx_n
set_property LOC A4 [get_ports {pcie_x2_tx_n[1]}]

| A6         | pcie_x2_tx_n[1] |            | MGTPTXN2_216                 | OUTPUT        |
| A10        | pcie_x2_rx_n[1] |            | MGTPRXN2_216                 | INPUT         |
| B6         | pcie_x2_tx_p[1] |            | MGTPTXP2_216                 | OUTPUT        |
| B10        | pcie_x2_rx_p[1] |            | MGTPRXP2_216                 | INPUT         |
| C7         | pcie_x2_tx_n[0] |            | MGTPTXN3_216                 | OUTPUT        |
| C9         | pcie_x2_rx_n[0] |            | MGTPRXN3_216                 | INPUT         |

[luigifcruz]

Looks like I found the problem. The pci_s7_PCIE_X0Y0.xdc file was generated with the correct transceiver channels but the pipe_lane[X] was incorrect. I changed the lanes to the desired values and everything generated and worked as intended. I don't know how to change Litex code to ensure this doesn't happen again. Any help would be appreciated.

# PCIe Lane 0
set_property LOC GTPE2_CHANNEL_X0Y7 [get_cells {inst/gt_top_i/pipe_wrapper_i/pipe_lane[0].gt_wrapper_i/gtp_channel.gtpe2_channel_i}]
# PCIe Lane 1
set_property LOC GTPE2_CHANNEL_X0Y6 [get_cells {inst/gt_top_i/pipe_wrapper_i/pipe_lane[1].gt_wrapper_i/gtp_channel.gtpe2_channel_i}]

# PCIe Lane 0
set_property LOC GTPE2_CHANNEL_X0Y7 [get_cells {inst/gt_top_i/pipe_wrapper_i/pipe_lane[3].gt_wrapper_i/gtp_channel.gtpe2_channel_i}]
# PCIe Lane 1
set_property LOC GTPE2_CHANNEL_X0Y6 [get_cells {inst/gt_top_i/pipe_wrapper_i/pipe_lane[0].gt_wrapper_i/gtp_channel.gtpe2_channel_i}]

| A4         | pcie_x2_tx_n[1] |            | MGTPTXN0_216                 | OUTPUT        |
| A6         | pcie_x2_tx_n[0] |            | MGTPTXN2_216                 | OUTPUT        |
| A8         | pcie_x2_rx_n[1] |            | MGTPRXN0_216                 | INPUT         |
| A10        | pcie_x2_rx_n[0] |            | MGTPRXN2_216                 | INPUT         |

[enjoy-digital]

Hi @luigifcruz,

sorry for the late answer. This is indeed the correct fix to force Vivado to use the correct transceiver channel when only one or two channel of the transceiver quad are used. For now I haven't found a better solution and to have this in a LiteX design, I do it like this:

platform.toolchain.pre_placement_commands.append("reset_property LOC [get_cells -hierarchical -filter {{NAME=~pcie_support/*gtp_channel.gtpe2_channel_i}}]")
platform.toolchain.pre_placement_commands.append("set_property LOC GTPE2_CHANNEL_X0Y7 [get_cells -hierarchical -filter {{NAME=~pcie_support/*gtp_channel.gtpe2_channel_i}}]")

Note: that if you are not using the other transceiver of the quad for other purpose, it's perfecly fine to keep the link in Gen2 X4 mode: The FPGA has no trouble handling this link speed and the PHY/Core will automatically fallback to X2 or X1 lanes if that's what is physically available on the M2 slot.

[luigifcruz]

Hey @enjoy-digital,

Thanks for chiming in! I didn’t know about the fallback! This is great since I don’t need to change the number of lanes in the gateware.