Configurable hw socket table size

[quetric]

Modified Makefile and XRT driver to support a configurable hardware socket table, as the existing hard-coded size is too small for some applications.

[mariodruiz]

@quetric let me know when this is tested and I can merge.

[lizajoseph]

Hi Mario,

I came across this project as I am looking for example designs to get started on my Xilinx Alveo u50 accelerator card and the data transfer between the host and card is via the Ethernet ports. Just wanted to know if I can try out this design available from Github and implement this design on my alveo u50 card to get started?

[mariodruiz]

Hi @lizajoseph,

Yes, you can use your U50. There's a table with the supported boards and associated shell. https://github.com/Xilinx/xup_vitis_network_example#alveo-cards

However, your question is not related to this PR. Please, open an issue or use existing (related) issues.

[lizajoseph]

Hi,

Thank you for your response, As I am reading through the example design, I see this Requirements section as mentioned below. Do I need to purchase this license in order to implement and run this design? Requirements In order to generate this design you will need a valid UltraScale+ Integrated 100G Ethernet Subsystemhttps://www.xilinx.com/products/intellectual-property/cmac_usplus.html license set up in Vivado.

Thanks, Liza From: Mario Ruiz @.> Sent: Friday, November 24, 2023 4:15 PM To: Xilinx/xup_vitis_network_example @.> Cc: Liza Joseph @.>; Mention @.> Subject: Re: [Xilinx/xup_vitis_network_example] Configurable hw socket table size (PR #117)

Hi @lizajosephhttps://eu-central-1.protection.sophos.com?d=github.com&u=aHR0cHM6Ly9naXRodWIuY29tL2xpemFqb3NlcGg=&i=NjUxY2UxZGIzOWRmODEzYjEzZTdlZmU4&t=dVIwL20xd1ZqVEdkQSt6TmpEa1hJNDgvdzhzdmZWc0YxNkpFOXhmR215TT0=&h=c485a92774344b628298b9cab8b69d03&s=AVNPUEhUT0NFTkNSWVBUSVY6K0T94M0wPyDMzKN9BocAHrXlbSFJ3THbbw-AzQ3TEw,

Yes, you can use your U50. There's a table with the supported boards and associated shell. https://github.com/Xilinx/xup_vitis_network_example#alveo-cardshttps://eu-central-1.protection.sophos.com?d=github.com&u=aHR0cHM6Ly9naXRodWIuY29tL1hpbGlueC94dXBfdml0aXNfbmV0d29ya19leGFtcGxlI2FsdmVvLWNhcmRz&i=NjUxY2UxZGIzOWRmODEzYjEzZTdlZmU4&t=VmZGTXQ1Z0lGdlJjTjVJNDI5NGxWaDNUWVJEN21XL1U0cDJlMW1WMkI5VT0=&h=c485a92774344b628298b9cab8b69d03&s=AVNPUEhUT0NFTkNSWVBUSVY6K0T94M0wPyDMzKN9BocAHrXlbSFJ3THbbw-AzQ3TEw

However, your question is not related to this PR. Please, open an issue or use existing (related) issues.

— Reply to this email directly, view it on GitHubhttps://eu-central-1.protection.sophos.com?d=github.com&u=aHR0cHM6Ly9naXRodWIuY29tL1hpbGlueC94dXBfdml0aXNfbmV0d29ya19leGFtcGxlL3B1bGwvMTE3I2lzc3VlY29tbWVudC0xODI1NDg0NTgy&i=NjUxY2UxZGIzOWRmODEzYjEzZTdlZmU4&t=MG1vME5hNXFkWHU5TURkZmZoM3FYclNXeFhwaC83c3E0TW1PVkZ1V3hMcz0=&h=c485a92774344b628298b9cab8b69d03&s=AVNPUEhUT0NFTkNSWVBUSVY6K0T94M0wPyDMzKN9BocAHrXlbSFJ3THbbw-AzQ3TEw, or unsubscribehttps://eu-central-1.protection.sophos.com?d=github.com&u=aHR0cHM6Ly9naXRodWIuY29tL25vdGlmaWNhdGlvbnMvdW5zdWJzY3JpYmUtYXV0aC9CRUE3U0xMNDNGUUVKNVIyWFY3UUpIRFlHQjNEWEFWQ05GU002QUFBQUFBNkc3Sk00R1ZISTJEU01WUVdJWDNMTVY0M09TTFRPTjJXS1EzUE5WV1dLM1RVSE1ZVFFNUlZHUTRESU5KWUdJ&i=NjUxY2UxZGIzOWRmODEzYjEzZTdlZmU4&t=Qk8vbGN2WFdQQVJNMWhrNUo1Yk83OVQzTERhRmZQMm5QU0RmV3U5bEFIMD0=&h=c485a92774344b628298b9cab8b69d03&s=AVNPUEhUT0NFTkNSWVBUSVY6K0T94M0wPyDMzKN9BocAHrXlbSFJ3THbbw-AzQ3TEw. You are receiving this because you were mentioned.Message ID: @.**@.>>

[Mellich]

The changes in this PR do not work for me. I tested this PR on our system with the following configuration:

• Alveo U280 (shell: xilinx_u280_gen3x16_xdma_1_202211_1)
• XRT 2.14
• Vivado/Vitis 2022.2
• Maximum sockets 16, 48, 60

I synthesized the basic kernels mm2s and s2mm as well as INTERFACE=3 mode to get one replication per QSFP port. For evaluation, I directly connected the QSFP ports of a single FPGA and wrote a simple test utilizing the basic kernels.

You can find all changes in this branch: https://github.com/Mellich/xup_vitis_network_example/tree/test-config-socket-table

In the tests, I send 2 GB of data from QSFP0 to QSFP1. The tests hang for socket sizes 48 and 60. Using the VNx C++ driver statistics, I generated the logs below.

MAX_SOCKETS=16 results are as expected:

Statistics after ARP and before tests
QSFP0:
udp in bytes = 0
udp app in bytes = 0
udp app out bytes = 0
udp out bytes = 0
ethhi_out_bytes = 0
eth_out_bytes = 30720
QSFP1:
udp in bytes = 0
udp app in bytes = 0
udp app out bytes = 0
udp out bytes = 0
ethhi_out_bytes = 0
eth_out_bytes = 30720

Statistics after 2 GB sent 0 -> 1
QSFP0:
udp in bytes = 0
udp app in bytes = 0
udp app out bytes = 2147483648
udp out bytes = 2190189360
ethhi_out_bytes = 2211542216
eth_out_bytes = 2211572936
QSFP1:
udp in bytes = 2190189360
udp app in bytes = 2147483648
udp app out bytes = 0
udp out bytes = 0
ethhi_out_bytes = 0
eth_out_bytes = 30720

MAX_SOCKETS 60:

Statistics after ARP and before tests
QSFP0:
udp in bytes = 0
udp app in bytes = 0
udp app out bytes = 0
udp out bytes = 0
ethhi_out_bytes = 0
eth_out_bytes = 30720
QSFP1:
udp in bytes = 0
udp app in bytes = 0
udp app out bytes = 0
udp out bytes = 0
ethhi_out_bytes = 0
eth_out_bytes = 30720

Statistics after 2 GB sent 0 -> 1
QSFP0:
udp in bytes = 0
udp app in bytes = 0
udp app out bytes = 2147483648
udp out bytes = 0
ethhi_out_bytes = 0
eth_out_bytes = 30720
QSFP1:
udp in bytes = 0
udp app in bytes = 0
udp app out bytes = 0
udp out bytes = 0
ethhi_out_bytes = 0
eth_out_bytes = 30720

So the data seems to get lost on the sending side within the UDP stack and before the Ethernet stack.

[quetric]

Hi @Mellich I pushed a fix - the UDP assigned aperture was too small and session valids weren't accessible beyond certain values for MAX_SOCKETS. Would you mind doing a clean rebuild of the xclbin and testing again?

[Mellich]

Thanks @quetric, next to the small test above I also did some testing with larger socket tables. Now the 60 sockets configuration works as expected.

[quetric]

Thanks @Mellich!

@mariodruiz this is now tested and can be merged