What should I connect to the AXI-Stream?

offnaria commented 5 months ago

Hello. I have a question about the usage of nanotube-generated HLS modules. I'd like to know what I should connect to the input and output of the AXI-Stream chain.

What you are trying to do.

I'm trying to implement examples/katran onto my Alveo U200 board.

What commands you typed.

I typed commands shown in the README.md (but I sat XILINX_VITIS_HLS to the Vitis's actual path, instead of the copied directory) and successfully built LLVM and nanotube. After that, I followed examples/katran/README.md and typed the following command to generate HLS modules for Alveo U200.

scripts/hls_build examples/katran/balancer_kern.O3.nt.req.lower.inline.platform.optreq.converge.pipeline.link_taps.inline_opt.hls tmp/katran -j$(($(nproc)/2+1)) -L/usr/lib/x86_64-linux-gnu -L/usr/lib/gcc/x86_64-linux-gnu/11 --part xcu200-fsgd2104-2-e

What output you saw.

I connected 80 of HLS modules on the Vivado's block design by following vitis_opts.ini, and I saw the huge AXI-Stream chain as shown below.

I saw unconnected 520-bits input/output interfaces.

Screenshots

![image](https://github.com/Xilinx/nanotube/assets/48970434/b3b41f20-57ef-48e2-a132-4a0eb747fa04) ![image](https://github.com/Xilinx/nanotube/assets/48970434/eb61482d-b20b-4016-a257-f327248b86b4)

I found in vitis_opts.ini that mae2p_kernel0 and p2vnr_kernel0 are suggested to connect, but what are they?

$ grep kernel examples/katran/balancer_kern.O3.nt.req.lower.inline.platform.optreq.converge.pipeline.link_taps.inline_opt.hls/vitis_opts.ini
sc=mae2p_kernel0:stage_8.port0
sc=stage_79.port1:p2vnr_kernel0

The commit ID of the Nanotube repository.

$ git log -1
commit 9ff2e920b02eb1b7df55811cf8b25f1dd410f61c (HEAD -> main, origin/main, origin/HEAD)
Author: Stephan Diestelhorst <stephand@xilinx.com>
Date:   Thu Apr 13 16:13:10 2023 +0100

    Testing: Remove redundant path normalisation

    Removing an additional call to os.path.normpath which is redundant
    because the previous call to os.path.abspath already performs the
    normalisation as part of its functionality.

The version of the operating system being run.

$ lsb_release -a
No LSB modules are available.
Distributor ID: Ubuntu
Description:    Ubuntu 22.04.4 LTS
Release:        22.04
Codename:       jammy
$ uname -r
5.15.146.1-microsoft-standard-WSL2

Thank you.

stephand commented 4 months ago

Hi @offnaria,

many thanks for trying out Nanotube and writing an excellent bug report!

You have been doing all steps correctly, and your result is expected: Nanotube expects a NIC shell that it can place the compiled plugin in. We have been using the SN1022 and X3 hybrid NIC shells, but they are not easily available.

For experimenting with your Alveo U200 board, the best way forward would be:

pick a NIC shell that is available (https://github.com/Xilinx/open-nic should be a good starting point)
add support for its bus formats, similar to the code in libnt/nanotube_packet_metadata_*.cpp and libnt/nanotube_packet_taps_*.cpp
connect the input / output busses from the NIC shell to the code generated by Nanotube
success :)

The idea is that Nanotube is independent (largely) of the actual bus formats that are used for the plugin interfaces in the NIC shell. Instead, these wrappers provide ways to detect end-of-packet / packet length, and how wide the bus is etc.

It would be great to add support for open-nic to Nanotube, so please let me know if you would like to have a look into that. We can support you and accept pull requests :)

offnaria commented 4 months ago

Hi @stephand

Thank you so much for providing step-by-step lecture. Adding support for open-nic and so on sounds interesting but challenging. I'll ask you again when I have enough time to work on it.

Anyway, thank you again.

ironmanna commented 3 months ago

Hi @offnaria ,

I'm Simone Mannarino, a master's student at Politecnico di Milano and the University of Illinois at Chicago. I'm working on a thesis titled “Exploring eBPF Capabilities with Hardware Acceleration on FPGAs for High-Performance Networking," which is similar to the project of implementing Katran on an Alveo U200 board. I've reproduced the steps targeting a U250 board.

It looks like we’re working on pretty similar things. I think it would be great if we could collaborate and share insights. Would you be interested in discussing our projects to explore potential collaboration?

Thanks!

offnaria commented 3 months ago

Hi @ironmanna

Thank you for comment. Do you mean you have ported Nanotube to open-nic on Alveo U250?

By the way, I'm not sure we can continue this discussion here...

ironmanna commented 3 months ago

I'm currently working on porting Nanotube to an Alveo U250. For further discussion, feel free to contact me at simone.mannarino@mail.polimi.it

Xilinx / nanotube