search

alexforencich / cocotbext-axi

AXI interface modules for Cocotb
MIT License
212 stars 70 forks source link

Simulation gets stuck with SIM=verilator 4.106 #35

Open andreaskuster opened 2 years ago

andreaskuster commented 2 years ago

Thanks a lot for this great project! @alexforencich @rodrigomelo9

I am trying to run the axi-verilog designs using verilator, however, this library does not seem to be compatible with verilator. I use version 4.106 which should be compatible with cocotb (all their base examples work).

However, it seems as if the simulation tests are stuck here: https://github.com/alexforencich/cocotbext-axi/blob/master/cocotbext/axi/axi_master.py#L413

Commands for reproduction:

python3 -m pip install cocotbext-axi==0.1.18
git clone https://github.com/alexforencich/cocotbext-axi.git
cd cocotbext-axi/tests/axi/
git checkout v0.1.18
SIM=verilator make

running make without SIM=verilator works

The log for the axi test is:

make -f Makefile results.xml
make[1]: Entering directory '/home/andreas/Desktop/cocotbext-axi/tests/axi'
MODULE=test_axi TESTCASE= TOPLEVEL=test_axi TOPLEVEL_LANG=verilog \
         sim_build/Vtop 
     -.--ns INFO     cocotb.gpi                         ..mbed/gpi_embed.cpp:76   in set_program_name_in_venv        Did not detect Python virtual environment. Using system-wide Python interpreter
     -.--ns INFO     cocotb.gpi                         ../gpi/GpiCommon.cpp:99   in gpi_print_registered_impl       VPI registered
     0.00ns INFO     Running on Verilator version 4.106 2020-12-02
     0.00ns INFO     Running tests with cocotb v1.6.1 from /home/andreas/.local/lib/python3.8/site-packages/cocotb
     0.00ns INFO     Seeding Python random module with 1642601669
     0.00ns INFO     Found test test_axi.run_test_write_001
     0.00ns INFO     Found test test_axi.run_test_write_002
     0.00ns INFO     Found test test_axi.run_test_write_003
     0.00ns INFO     Found test test_axi.run_test_write_004
     0.00ns INFO     Found test test_axi.run_test_write_005
     0.00ns INFO     Found test test_axi.run_test_write_006
     0.00ns INFO     Found test test_axi.run_test_write_007
     0.00ns INFO     Found test test_axi.run_test_write_008
     0.00ns INFO     Found test test_axi.run_test_write_009
     0.00ns INFO     Found test test_axi.run_test_write_010
     0.00ns INFO     Found test test_axi.run_test_write_011
     0.00ns INFO     Found test test_axi.run_test_write_012
     0.00ns INFO     Found test test_axi.run_test_read_001
     0.00ns INFO     Found test test_axi.run_test_read_002
     0.00ns INFO     Found test test_axi.run_test_read_003
     0.00ns INFO     Found test test_axi.run_test_read_004
     0.00ns INFO     Found test test_axi.run_test_read_005
     0.00ns INFO     Found test test_axi.run_test_read_006
     0.00ns INFO     Found test test_axi.run_test_read_007
     0.00ns INFO     Found test test_axi.run_test_read_008
     0.00ns INFO     Found test test_axi.run_test_read_009
     0.00ns INFO     Found test test_axi.run_test_read_010
     0.00ns INFO     Found test test_axi.run_test_read_011
     0.00ns INFO     Found test test_axi.run_test_read_012
     0.00ns INFO     Found test test_axi.run_test_write_words_001
     0.00ns INFO     Found test test_axi.run_test_read_words_001
     0.00ns INFO     Found test test_axi.run_stress_test_001
     0.00ns INFO     running run_test_write_001 (1/27)
     0.00ns INFO     AXI master (write)
     0.00ns INFO     cocotbext-axi version 0.1.18
     0.00ns INFO     Copyright (c) 2020 Alex Forencich
     0.00ns INFO     https://github.com/alexforencich/cocotbext-axi
     0.00ns INFO     Reset de-asserted
     0.00ns INFO     Reset de-asserted
     0.00ns INFO     Reset de-asserted
     0.00ns INFO     AXI master configuration:
     0.00ns INFO       Address width: 32 bits
     0.00ns INFO       ID width: 8 bits
     0.00ns INFO       Byte size: 8 bits
     0.00ns INFO       Data width: 32 bits (4 bytes)
     0.00ns INFO       Max burst size: 2 (4 bytes)
     0.00ns INFO       Max burst length: 256 cycles (1024 bytes)
     0.00ns INFO     AXI master signals:
     0.00ns INFO       awaddr width: 32 bits
     0.00ns INFO       awburst width: 2 bits
     0.00ns INFO       awcache width: 4 bits
     0.00ns INFO       awid width: 8 bits
     0.00ns INFO       awlen width: 8 bits
     0.00ns INFO       awlock width: 1 bits
     0.00ns INFO       awprot width: 3 bits
     0.00ns INFO       awqos width: 4 bits
     0.00ns INFO       awready width: 1 bits
     0.00ns INFO       awregion width: 4 bits
     0.00ns INFO       awsize width: 3 bits
     0.00ns INFO       awuser width: 1 bits
     0.00ns INFO       awvalid width: 1 bits
     0.00ns INFO       wdata width: 32 bits
     0.00ns INFO       wlast width: 1 bits
     0.00ns INFO       wready width: 1 bits
     0.00ns INFO       wstrb width: 4 bits
     0.00ns INFO       wuser width: 1 bits
     0.00ns INFO       wvalid width: 1 bits
     0.00ns INFO       bid width: 8 bits
     0.00ns INFO       bready width: 1 bits
     0.00ns INFO       bresp width: 2 bits
     0.00ns INFO       buser width: 1 bits
     0.00ns INFO       bvalid width: 1 bits
     0.00ns INFO     Reset de-asserted
     0.00ns INFO     AXI master (read)
     0.00ns INFO     cocotbext-axi version 0.1.18
     0.00ns INFO     Copyright (c) 2020 Alex Forencich
     0.00ns INFO     https://github.com/alexforencich/cocotbext-axi
     0.00ns INFO     Reset de-asserted
     0.00ns INFO     Reset de-asserted
     0.00ns INFO     AXI master configuration:
     0.00ns INFO       Address width: 32 bits
     0.00ns INFO       ID width: 8 bits
     0.00ns INFO       Byte size: 8 bits
     0.00ns INFO       Data width: 32 bits (4 bytes)
     0.00ns INFO       Max burst size: 2 (4 bytes)
     0.00ns INFO       Max burst length: 256 cycles (1024 bytes)
     0.00ns INFO     AXI master signals:
     0.00ns INFO       araddr width: 32 bits
     0.00ns INFO       arburst width: 2 bits
     0.00ns INFO       arcache width: 4 bits
     0.00ns INFO       arid width: 8 bits
     0.00ns INFO       arlen width: 8 bits
     0.00ns INFO       arlock width: 1 bits
     0.00ns INFO       arprot width: 3 bits
     0.00ns INFO       arqos width: 4 bits
     0.00ns INFO       arready width: 1 bits
     0.00ns INFO       arregion width: 4 bits
     0.00ns INFO       arsize width: 3 bits
     0.00ns INFO       aruser width: 1 bits
     0.00ns INFO       arvalid width: 1 bits
     0.00ns INFO       rdata width: 32 bits
     0.00ns INFO       rid width: 8 bits
     0.00ns INFO       rlast width: 1 bits
     0.00ns INFO       rready width: 1 bits
     0.00ns INFO       rresp width: 2 bits
     0.00ns INFO       ruser width: 1 bits
     0.00ns INFO       rvalid width: 1 bits
     0.00ns INFO     Reset de-asserted
     0.00ns INFO     AXI slave model (write)
     0.00ns INFO     cocotbext-axi version 0.1.18
     0.00ns INFO     Copyright (c) 2021 Alex Forencich
     0.00ns INFO     https://github.com/alexforencich/cocotbext-axi
     0.00ns INFO     Reset de-asserted
     0.00ns INFO     Reset de-asserted
     0.00ns INFO     Reset de-asserted
     0.00ns INFO     AXI slave model configuration:
     0.00ns INFO       Address width: 32 bits
     0.00ns INFO       ID width: 8 bits
     0.00ns INFO       Byte size: 8 bits
     0.00ns INFO       Data width: 32 bits (4 bytes)
     0.00ns INFO     AXI slave model signals:
     0.00ns INFO       awaddr width: 32 bits
     0.00ns INFO       awburst width: 2 bits
     0.00ns INFO       awcache width: 4 bits
     0.00ns INFO       awid width: 8 bits
     0.00ns INFO       awlen width: 8 bits
     0.00ns INFO       awlock width: 1 bits
     0.00ns INFO       awprot width: 3 bits
     0.00ns INFO       awqos width: 4 bits
     0.00ns INFO       awready width: 1 bits
     0.00ns INFO       awregion width: 4 bits
     0.00ns INFO       awsize width: 3 bits
     0.00ns INFO       awuser width: 1 bits
     0.00ns INFO       awvalid width: 1 bits
     0.00ns INFO       wdata width: 32 bits
     0.00ns INFO       wlast width: 1 bits
     0.00ns INFO       wready width: 1 bits
     0.00ns INFO       wstrb width: 4 bits
     0.00ns INFO       wuser width: 1 bits
     0.00ns INFO       wvalid width: 1 bits
     0.00ns INFO       bid width: 8 bits
     0.00ns INFO       bready width: 1 bits
     0.00ns INFO       bresp width: 2 bits
     0.00ns INFO       buser width: 1 bits
     0.00ns INFO       bvalid width: 1 bits
     0.00ns INFO     Reset de-asserted
     0.00ns INFO     AXI slave model (read)
     0.00ns INFO     cocotbext-axi version 0.1.18
     0.00ns INFO     Copyright (c) 2021 Alex Forencich
     0.00ns INFO     https://github.com/alexforencich/cocotbext-axi
     0.00ns INFO     Reset de-asserted
     0.00ns INFO     Reset de-asserted
     0.00ns INFO     AXI slave model configuration:
     0.00ns INFO       Address width: 32 bits
     0.00ns INFO       ID width: 8 bits
     0.00ns INFO       Byte size: 8 bits
     0.00ns INFO       Data width: 32 bits (4 bytes)
     0.00ns INFO     AXI slave model signals:
     0.00ns INFO       araddr width: 32 bits
     0.00ns INFO       arburst width: 2 bits
     0.00ns INFO       arcache width: 4 bits
     0.00ns INFO       arid width: 8 bits
     0.00ns INFO       arlen width: 8 bits
     0.00ns INFO       arlock width: 1 bits
     0.00ns INFO       arprot width: 3 bits
     0.00ns INFO       arqos width: 4 bits
     0.00ns INFO       arready width: 1 bits
     0.00ns INFO       arregion width: 4 bits
     0.00ns INFO       arsize width: 3 bits
     0.00ns INFO       aruser width: 1 bits
     0.00ns INFO       arvalid width: 1 bits
     0.00ns INFO       rdata width: 32 bits
     0.00ns INFO       rid width: 8 bits
     0.00ns INFO       rlast width: 1 bits
     0.00ns INFO       rready width: 1 bits
     0.00ns INFO       rresp width: 2 bits
     0.00ns INFO       ruser width: 1 bits
     0.00ns INFO       rvalid width: 1 bits
     0.00ns INFO     Reset de-asserted
     2.00ns INFO     Reset asserted
     2.00ns INFO     Reset asserted
     2.00ns INFO     Reset asserted
     2.00ns INFO     Reset asserted
     2.00ns INFO     Reset asserted
     2.00ns INFO     Reset asserted
     2.00ns INFO     Reset asserted
     2.00ns INFO     Reset asserted
     2.00ns INFO     Reset asserted
     2.00ns INFO     Reset asserted
     2.00ns INFO     Reset asserted
     2.00ns INFO     Reset asserted
     2.00ns INFO     Reset asserted
     2.00ns INFO     Reset asserted
     2.00ns INFO     Reset de-asserted
     2.00ns INFO     Reset de-asserted
     2.00ns INFO     Reset de-asserted
     2.00ns INFO     Reset de-asserted
     2.00ns INFO     Reset de-asserted
     2.00ns INFO     Reset de-asserted
     2.00ns INFO     Reset de-asserted
     2.00ns INFO     Reset de-asserted
     2.00ns INFO     Reset de-asserted
     2.00ns INFO     Reset de-asserted
     2.00ns INFO     Reset de-asserted
     2.00ns INFO     Reset de-asserted
     2.00ns INFO     Reset de-asserted
     2.00ns INFO     Reset de-asserted
    10.00ns INFO     length 1, offset 0
    10.00ns INFO     Write start addr: 0x00001000 awid: 0x0 prot: AxiProt.NONSECURE data: 00
    10.00ns INFO     Write burst start awid: 0x0 awaddr: 0x00001000 awlen: 0 awsize: 2 awprot: AxiProt.NONSECURE
rodrigomelo9 commented 2 years ago

Hi @andreaskuster Only to clarify, I am simply a recent user of cocotbext-axi.

I can't right now, but I will try to reproduce it and let you know. Let me say that I ran verilator --version in my Ubuntu 20.04 and it reported Verilator 4.028 2020-02-06 rev v4.026-92-g890cecc1 (are you sure that you are trying 1.106? seems extremely old).

andreaskuster commented 2 years ago

Thanks for your reply. And yes you were right, i meant version 4.106 (fixed it in the description), and the compatibility issue with newer versions i mentioned is according to the cocotb github issues and the comment here: https://docs.cocotb.org/en/stable/simulator_support.html

Furthermore, i tested different (older) versions of cocotbext-axi, with the same behaviour, so this problem seems to be present for a while.

rodrigomelo9 commented 2 years ago

Ok, I didn't know about the cocotb limitation using Verilator. I generally use cocotb with GHDL or iverilog.

andreaskuster commented 2 years ago

Sure, ghdl and iverilog are fine, as long as you work with vhdl or verilog. However, I am currently extending a project written in SystemVerilog, which is why I would like to get verilator up to speed.

Well, it is not really a limitation by design, but rather the devil that comes with the fact that verilator is extending the VPI infrastructure. This should eventually become compatible again.

However, I am not sure if verilator is really to blame here, since the latest (v1.6.1) cocotb basic examples seem to work with verilator v4.106. However, it is difficult for me to debug the issue, since this co-routine based code seems to be stuck in an infinity loop, but I do not understand the cause yet. @alexforencich Please let me know if you have any insights or clues that I can help with.

alexforencich commented 2 years ago

I'm not sure what the problem is here. But I'm also not going to do anything with verilator until https://github.com/verilator/verilator/issues/2778 is resolved so that cocotb will work correctly with current versions of verilator.

andreaskuster commented 2 years ago

Yeah sure I can understand that. Thanks for your clarification. Let's keep this open and come back as soon as verilator has been fixed.

BTW: cocotbext-axi + questasim works well for systemverilog.

alexforencich commented 2 years ago

I mean, it's also possible that I introduced some sort of bug recently that only affects verilator. So it's probably still worth getting to the bottom of what's going on here. Previously, I think I have tested this on verilator 4.106, and it worked fine. But there have been some changes since then, although I can't think of any change in particular that might have affected verilator specifically.

andreaskuster commented 2 years ago

I think that the problem either exists for a longer time, or it is already broken in verilator 4.106, since I tried different, older version combinations of cocotbext-axi and cocotb, but with the very same result of getting stuck.

SabraO commented 2 years ago

@andreaskuster I was also facing this issue, where after I upgraded several libraries the simulation for my previously working project got stuck. I'm using Verilator 4.106, and was using a previous release of cocotb and cocotbext-axi. It turns out the issue was with cocotb-bus, where earlier I was using version 0.1.1 and after the upgrade, it was 0.2.1.

Reverting cocotb-bus to version 0.1.1worked for me. And this works with even the newest cocotb and cocotb-ext versions.

pip3 install cocotb-bus==0.1.1

When a bus version > 0.1.1 is used the reset never gets asserted. I don't know why reset is affected. But hopefully, this solution helps.

andreaskuster commented 2 years ago

Thanks @SabraO, this indeed did the trick!

I opened a PR #37 to fix this.