ipbus / ipbus-firmware

Firmware that implements a reliable high-performance control link for particle physics electronics, based on the IPbus protocol
https://ipbus.web.cern.ch
Other
39 stars 31 forks source link

IPBus options with Zynq #228

Open aranzaba opened 6 months ago

aranzaba commented 6 months ago

Dear IPBus developers,

I am working at the European Synchrotron (ESRF) in Grenoble, developing electronics for instrumentation and data acquisition systems. I had the opportunity to learn and use IPBus during the years I was working at CERN, and now I would like to start using it in some of our projects at the ESRF.

Right now, I am evaluating what would be the best way to use IPBus with an evaluation platform from Enclustra (Mercury PE3 base board + Mercury XU8 SOM with a Zynq US+ MPSoC). I have been digging through IPBus documentation and trying to understand what would be the options that I have to use IPBus with this system.

A nice option would be to read/write registers in the PL using one of the two available RJ45 connectors in the PE3 base board. The RJ45 are connected to a TI RGMII PHY that goes to the Ethernet MAC (GEM) Controller in the PS (see the link below). Then, the GEM is accessible from the PL, so I was wondering if it is currently supported to connect IPBus to the GEM for accessing the registers in the FPGA without involving the ARM processors. Xilinx-Wiki, Zynq MPSoC Ethernet examples

Another option that I see could be to follow the ZCU102 AXI chip-to-chip example design, however there is a part that is not so clear to me. With this example design I am able to access the registers from the Linux running in the processor, but would it be possible to access the registers from an external PC via 1 GbE link using uHAL library? Linux drivers?

And finally, the third option that I am contemplating is to forget about the RJ45 in the PE3 base board and to connect a SFP-to-RJ45 converter module to one the SFP cages available, as I am not interested to use a NIC for this project. But in this case it would be a pity not making use of the already available RJ45 links in the base board.

I would deeply appreciate your feedback or whether if you could provide point me to any reference design with a similar user cases.

Many thanks,

Nordin Aranzabal

tswilliams commented 6 months ago

In terms of reference/example designs, I think you've already found the most relevant one to this use case - the ZCU102 AXI chip-to-chip example design.

Another option that I see could be to follow the ZCU102 AXI chip-to-chip example design, however there is a part that is not so clear to me. With this example design I am able to access the registers from the Linux running in the processor, but would it be possible to access the registers from an external PC via 1 GbE link using uHAL library? Linux drivers?

Technically speaking, you could have two transport interfaces connected up to the IPbus transactor - one for the Zynq PL-PS interface, and the other for the UDP interface. But in terms of system architecture, I think using both interfaces would typically have more disadvantages than advantages (e.g. confusion of what registers are being read/written from one interface vs the other).

From my perspective, one key question to ask is whether or not you expect to have software running on the Zynq PS. If you do, and you'd want that software to interace with firmware in the PL, then I would suggest just using the Zynq PL-PS transport interface, hence using uHAL Zynq PS. If you will not have software running on the Zynq PS, then you could just bypass it with either the RJ45 or SFP connections as you outlined.

I hope that was helpful. Let me know if you have more questions.

aranzaba commented 6 months ago

Thanks a lot Tom.

Technically speaking, you could have two transport interfaces connected up to the IPbus transactor - one for the Zynq PL-PS interface, and the other for the UDP interface. But in terms of system architecture, I think using both interfaces would typically have more disadvantages than advantages (e.g. confusion of what registers are being read/written from one interface vs the other).

Just to be sure I understood it correctly. When using a Zynq PL-PS transport interface, it is not possible to access registers in the PL from an external machine using IPBus software, e.g. external machine running IPBus software => Zynq PS => PL? The Zynq PS can access registers by running uHAL in an embedded Linux, but in this case we would need another channel and application (something different to IPBus) to access PL registers from an external device, right?

Is there any example design on how to connect an IPBus transactor from the PL to the Ethernet MAC (GEM) Controller ) in the PS (GMII via EMIO)? If there is not any example for this use case, could you please provide more information on how to connect an IPBus transactor using this interface?