Integration in behavioral/RTL simulation through Vivado

Hello.

I have been using LibSystemCTLM-SoC to cosimulate my Zynq designs for behavioral/RTL simulations.

I use generated simulation scripts and files from Vivado and replace the "processing_system7_0_0" entity during elaboration (which normally is the Zynq Verification IP) by a custom made generic Zynq wrapper built around LibSystemCTLM-SoC and the TLM bridges. This allows me to CoSimulate the entire system with QuestaSim and QEMU. This required however to create several wrappers and scripts, especially to get all the parameters for the Zynq processing system (which ports are enabled, what are the width of the AXI ports etc.) as well as the creation of a specific SystemC Zynq model to instantiate all the ports to communicate with QEMU through remote port and convert the TLM transactions to AXI (Master/Slave) using the TLM bridges from LibSystemCTLM-SoC.

This works well but results in a rather large extra set of files and relies on home-made scripts (which will extract the parameters automatically from the IP files etc.). to configure everything.

It seems there could be some support in Vivado for automating this, but I could not find any documentation. In a Vivado project with a Zynq PS there are the following files under : project/project.srcs/sources_1/bd/block_design_name_bd/ip/block_design_name_bd_processing_system7_0_0/sim/

block_design_name_bd_processing_system7_0_0.sv
block_design_name_bd_processing_system7_0_0.v
libremoteport.so

So there seems to be a SystemVerilog entity which uses Remote Port and TLM through the DPI. Can this entity be used for co-simulation in any way ? Is there any documentation ? libremoteport.so offers functions to synchronize with QEMU as well as read and write remote port commands (very similar to https://github.com/Xilinx/libsystemctlm-soc/blob/master/libremote-port/remote-port-tlm.cc) however I did not find any documentation as to how to set this up. Is there any ?

LibSystemCTLM-SoC provides everything to create a "drop-in" replacement for the Zynq PS VIP used in simulation under Vivado which would allow to CoSimulate everything, is there anything provided here by Xilinx to launch such a simulation ?

Thank you for your time.

Hi Rick,

It's really impressive and cool what you've got going there. Do you think that there are any parts (wrappers, changes to libsystemctlm-soc, etc) of your work that you could share and that would make sense to contribute to either libsystemctlm-soc, systemctlm-cosim-demo? or perhaps to create a new github project that uses libsystemc-tlm-soc to demonstrate your setup? I'm sure others would find it useful.

With regards to Vivado, yes, both Vivado and Vitis have a co-simulation infrastructure that uses QEMU and a modified version libsystemctlm-soc. I'm not super familiar with how it all works there but I believe it's fully integrated with Vitis platforms, you just need to select QEMU as the target to run on. For Vivado designs, I'm guessing you need to export the Hardware project with simulator and import it in Vitis to run software on it. It can also be done with command-lines tools/scripts. I've only tried it with simple examples on the Vitis GUI so I'll have to refer you to Xilinx support for the details. I'll see if I can find any pointers to documentation.

Best regards, Edgar

Thank you for your response.

I have to try out Vitis, I tend to stick with a version of Vivado that works with my current projects because the installation is quite large and takes time to do but I will try Vitis when I have some time.

You could use what I made (or part thereof) to create an example design if your like.

For the SystemC part it is publicly available in https://github.com/rick-heig/zynq7-cosim under src_sc/zynq7_ps.h and src_sc/zynq7_ps.cc

It is similar to your examples but allows to enable M_AXI_GP0-1, S_AXI_GP0-1, S_AXI_HP0-3 and the clocks (and frequencies) based on generic parameters. The widths of the AXI signals must be passed by the preprocessor because the C++ templates used for the AXI interfaces require a constant for their specialization and cannot rely on this->my_generic_constant, they must be known at compile time. It also has support for the IRQs from PL to PS but not the other way around for now (easy to add).

The other files :

src_vhdl/zynq7_ps_wrapper.vhd is a wrapper that sets default values for input ports so that it is possible to instantiate this wrapper without connecting everything.
src_vhdl/block_design_bd_processing_system7_0_0_template.vho is a template that is meant to be modified based on the project

The part that is not public for now is the code to extract all the parameters and create the compilation scripts for QuestaSim (compile everything needed from LibSystemCTLM-SoC) and configure the zynq7_ps module (it basically adds -D parameters to the compilation CXXFLAGS to define these sizes at compile time) and also compile the wrappers. It would also modifiy the .vho template to have all the ports to be a "drop-in" replacement for the ZYNQ7 VIP of the Vivado simulation. But this can all be done by hand. Once everything is compiled the generated and compiled block_design_bd_processing_system7_0_0 will simply replace the original one used in sim (Xilinx Zynq VIP), the other scripts to compile all the Vivado project files and IPs remain the same. I hope my explanation is understandable.

The file libSystemCTLM-SoC_wo_async_questa.patch is a patch for libSystemCTLM-SoC to make it compatible with QuestaSim, you can have a look at it, some interesting points :

QuestaSim does not support calls to async_attach_suspending(), async_detach_suspending(), therefore I needed to remove code that relied on async_event.h
vopt will actually call the constructors of the SystemC modules and does not support calls to exit() if I remember correctly it will already complain at compilation time. (some calls were just replaced by a for(;;); which may not be the most appropriate)
other modifications are just some casts for pointers

Note : Also since vopt already calls the constructor it requires a connection to QEMU (so QEMU needs to be launched once during optimization, then again for simulation)

I was trying to create a robust tool and documentation for my company in order to facilitate Zynq CoSimulation based on Vivado generated simulations but I don't want to do all this if Vivado or Vitis already enables this, but I could not find much information, I have been reading UG1144 and UG1169 and searched for other related documentation but could not find anything (well there is the wiki but it is a bit outdated https://xilinx-wiki.atlassian.net/wiki/spaces/A/pages/18842109/QEMU+SystemC+and+TLM+CoSimulation)

If you have any pointers to documentation it would be highly appreciated ! Thank you. Rick

Thanks Rick,

I'll try to find some time to go through your patch and see if we can find solutions to those problems in libsystemctlm-soc so that we can minimize the diff. If you'd like to propose patches/pull request, that would also be great.

I was looking at your code a little and noticed you're doing a lot of manual AXI signal connections. Not sure if it will work for you but you might find it interesting to look at signals-axi.h. It makes it easy to declare AXI signal sets and also to connect them to ports. https://github.com/Xilinx/libsystemctlm-soc/blob/master/tests/test-modules/signals-axi.h Here's an example: https://github.com/Xilinx/libsystemctlm-soc/blob/master/tests/rtl-bridges/axi/test-pcie-master-axi.cc

Regarding Vivado/Vitis documentation, I didn't find too much, it's surprisingly hard to find detailed information. Anyway, here are a few links that may be of interest (first one is probably the best):

This describes the creation of a platform in some detail allthough it doesn't talk too much about QEMU. Look at the "set_property SELECTED_SIM_MODEL tlm [get_bd_cells /zynq_ultra_ps_e_0]" https://www.xilinx.com/html_docs/xilinx2019_2/vitis_doc/Chunk373975992.html

https://www.xilinx.com/html_docs/xilinx2019_2/vitis_doc/Chunk468473335.html#jvn1570652701832 https://beetlebox.org/getting-started-with-computer-vision-for-vitis-embedded-systems-part-2/ https://forums.xilinx.com/t5/Adaptable-Advantage-Blog/MicroZed-Chronicles-Vitis-Emulation/ba-p/1069996 This one is a bit old for the SDSoC environment https://www.xilinx.com/support/documentation/sw_manuals/xilinx2018_3/ug1027-sdsoc-user-guide.pdf

Hello, thank you for your reply.

To be honest I never did any SystemC until a month or two ago in order to setup a "full-system" Co-Simulation environment for the Zynq-7000 based on QEMU + QuestaSim.

I understand that it might be better to use some kind of interface instead of connecting everything manually as was done in my SystemC file. I am used to SystemVerilog modports and this makes connecting AXI-Full busses very easy but did not know of an equivalent SystemC approach. I was already confused by the requirement that signals are 1-to-1 connections.

Probably it can be simplified in SystemC as well but I don't plan on changing this for the moment since it only results less code and the Zynq-7000 will ports will not change over time, so I might look into this if I do the Zynq-UltraScale someday.

As for the documentation / Vitis I read a ton of user guides, tutorials, forum posts etc. and could not find a good Linux "full-system" simulation example and could not manage to generate this myself from the tools (I tried several ways). I can use the IDE and connect to my instance of QEMU via TCF but was not able to use the QEMU that came with Vitis and have Vitis or Vivado generate the rest of the simulation scripts.

So I think I'll stay with a solution based on Xilinx QEMU (compiled e.g., via PetaLinux) with CoSimulation dtb and my own generated files to simulate the Zynq7 PS.

I will write a guide about how to setup this Co-Simulation, for our internal use, but it will also be publicly available.

I will try to set this up and if I have something running nicely I will probably do a pull request on some changes in the LibSystemCTLM-SoC repo. I will probably add some preprocessor directives #if __QUESTA__ or something the like to exclude async_event and change the exit(...);to return statements.

And in the mean time I hope some documentation on running software on QEMU from Vitis with PL simulation will come. Because from what I saw in the different Vitis/Vivado files and scripts everything needed should already be there to enable PS/PL CoSimulation I just could not manage to run it.

Hello,

I have written a step-by-step guide for setting up a Co-Simulation between Questasim and Qemu for simulating the whole PL of a Vivado Zynq RTL project while running Linux on Qemu.

It is available here : http://blog.reds.ch/?p=1180

All the required files and scripts are available open source under https://github.com/rick-heig/zynq7-cosim

I tried getting a Co-Simulation environment through Vitis and Xsim but could not manage to do so and did not find any guide except for the Vitis/SDSoC HLS/OpenCL HW-emu workflows. But I see that almost everything is already present in Vitis/Vivado to enable something similar to what I did. Maybe it's not ready yet.

Regards. Rick

Thank you very much Rick!

Impressive and interesting write up!

Cheers, Edgar

Xilinx / libsystemctlm-soc

Integration in behavioral/RTL simulation through Vivado #7