[TASK] [FPGA] Support for Xilinx AXI 1G/2.5 G Ethernet Subsystem

WorldofJARcraft commented 3 months ago

Is there an existing CVA6 task for this?

[X] I have searched the existing task issues

Task Description

I have a private fork of the cva6 project in which I have added hardware and software support (u-boot and Linux) for the Xilinx AXI 1G/2.5G Ethernet subsystem on the GenesysII FPGA board. I have testing booting with TFTP in u-boot and running SSH in Linux. Are you interested in a pull request?

Required Changes

build option to include Xilinx Ethernet instead of LowRISC Ethernet
modify existing build options from InclEthernet to InclLowRiscEthernet (and equivalent)
add scripts/Makefile entries / etc. for Xilinx Ethernet
add Xilinx Ethernet system to ariane_peripherals_xilinx
if Ethernet is included, add and connect AXI master buses for the Xilinx DMA controller
(this might be controversial) as AXI is not cache coherent, I currently disable the data cache when reading and modifying the DMA descriptors in memory in order to prevent the CPU from working with stale data in the cache (see also #2046). In order to be able to do this directly from Linux, I have added a build option to CVA6 that optionally allows the corresponding CSR to be written in supervisor mode. An alternative to this solution would be adding an SBI call for enabling and disabling the cache.

Current Status

I have a private fork of the project in which Ethernet works in u-boot and Linux.
There are some differences between my project and upstream cva6: E.g., I am using the Xilinx AXI Interconnect instead of the one from pulp-platform. Thus, if you are interested in a pull request, I will integrate my changes back into this repo, re-test and send a pull request.
I have patches that I can clean up and send to https://github.com/openhwgroup/cva6-sdk for software support.

Risks

The AXI Ethernet system is proprietary and you need a license for the underlying MAC core (TEMAC). You can generate a free 120-day evaluation license.
I am not sure if allowing the supervisor mode to access the cache management CSR has any security or conformity implications.

Prerequisites

No response

KPI (KEY Performance Indicators)

As the AXI Subsystem uses DMA, it should provide higher throughput than the LowRISC Ethernet subsystem. Also, it should reduce the amount of instructions/computation time required for networking.

Description of Done

An FPGA bitstream with the existing Ethernet stack can be generated
An FPGA bitstream with the Xilinx Ethernet stack can be generated
The Ethernet allows TFTP boot in u-boot
The Ethernet allows SSH in Linux

Associated PRs

No response

JeanRochCoulon commented 3 months ago

Thanks @WorldofJARcraft for your proposal. I do not work myself on FPGA, but @jquevremont would have an opinion on this.

jquevremont commented 3 months ago

Thanks @WorldofJARcraft. This delivers more Ethernet performance to CVA6-based processors. In the CVA6 project, we want to keep CVA6 as much vendor-independent as possible so this implementation cannot replace the existing one. However, I suggest that this is introduced as a new project concept to test the interest for OpenHW members. Do you work for an OpenHW member (Git nicknames do not make it obvious)?

WorldofJARcraft commented 3 months ago

Thanks @WorldofJARcraft. This delivers more Ethernet performance to CVA6-based processors. In the CVA6 project, we want to keep CVA6 as much vendor-independent as possible so this implementation cannot replace the existing one. However, I suggest that this is introduced as a new project concept to test the interest for OpenHW members. Do you work for an OpenHW member (Git nicknames do not make it obvious)?

No, my employer (publicly funded German research center) is not a member of the OpenHW group. I am working on a research project for which compatibility with proprietary (DMA) peripherals is crucial, and I am using CVA6 as CPU. This was the main motivation for integrating the Xilinx AXI Ethernet system with CVA6. I am confident I have solved all of the technical challenges in hardware and software, and I would like to give back to the project. I think it would be fairly easy to adapt the existing build system to make Xilinx Ethernet optional and mutually exclusive with the existing Ethernet, as code to disable the existing Ethernet already exists in the project (e.g., similar to the existing HAS_ETHERNET environment variable in the bootrom, I could introduce a HAS_XILINX_ETHERNET variable and use both of them for ifdef definitions in the verilog sources).

jquevremont commented 3 months ago

Could you join our next CVA6 meeting, on June 18th, 14:00-15:30 CEST?

WorldofJARcraft commented 3 months ago

Could you join our next CVA6 meeting, on June 18th, 14:00-15:30 CEST?

Yes, that works for me.

jquevremont commented 3 months ago

The Teams link is https://teams.microsoft.com/l/meetup-join/19%3ameeting_YzlhYTVjOGQtNTQ5Ny00YmFiLThkODQtMGE5MGU0YWRiMjA4%40thread.v2/0?context=%7b%22Tid%22%3a%226e603289-5e46-4e26-ac7c-03a85420a9a5%22%2c%22Oid%22%3a%22ff37ba72-0621-46b3-aff9-cf866855acc2%22%7d

The topic is confirmed on the agenda.

github-actions[bot] commented 2 months ago

👋 Hi there!

This issue seems inactive. Need more help? Feel free to update us. If there are no updates within the next few days, we'll go ahead and close this issue. 😊

jquevremont commented 2 months ago

Hi @WorldofJARcraft. What about adding a reference to your repo in the CVA6 README? (in a new section "related projects" before the acknowledgement section)?

WorldofJARcraft commented 2 months ago

Hi @WorldofJARcraft. What about adding a reference to your repo in the CVA6 README? (in a new section "related projects" before the acknowledgement section)?

Sounds good. I will need a few days to clean up my fork of the project, test it again and set it to public. I will notify you when this is done and send you the link.

WorldofJARcraft commented 2 months ago

Hello @jquevremont, my fork of the project is now online: https://github.com/cispa/CVA6-Vivado-Project-with-Xilinx-AXI-Ethernet/ In addition to the Xilinx Ethernet in hardware and software, I have added scripts for generating a fully-integrated Xilinx Vivado project. I hope this is useful for FPGA developers. I would be happy to join another meeting to discuss my changes in more detail if necessary.

github-actions[bot] commented 1 month ago

👋 Hi there!

This issue seems inactive. Need more help? Feel free to update us. If there are no updates within the next few days, we'll go ahead and close this issue. 😊

jquevremont commented 1 month ago

👋 Hi there!

This issue seems inactive. Need more help? Feel free to update us. If there are no updates within the next few days, we'll go ahead and close this issue. 😊

I am quite busy. I need to find time to create a "resources" section to mention this (and a few other).

jquevremont commented 5 days ago

First attempt: https://github.com/openhwgroup/cva6/pull/2514

WorldofJARcraft commented 4 days ago

Looks good to me, thank you.

openhwgroup / cva6