test_simple_processor fails

[BGull00]

When running SIMS=verilator WAVES=1 pytest tb/, test_simple_processor test fails on the main branch with the following output:

INFO cocotb: Running command: perl /usr/local/bin/verilator -cc --exe -Mdir /home/bryson/.cache/neuro_fpga/sims/verilator/test_simple_processor -DCOCOTB_SIM=1 --top-module basic_processor --vpi --public-flat-rw --prefix Vtop -o basic_processor -LDFLAGS -Wl,-rpath,/home/bryson/Documents/TENNLab/framework/pyframework/lib/python3.10/site-packages/cocotb/libs -L/home/bryson/Documents/TENNLab/framework/pyframework/lib/python3.10/site-packages/cocotb/libs -lcocotbvpi_verilator -Wno-fatal --coverage --trace-fst --trace-structs -I/home/bryson/Documents/TENNLab/framework/fpga/fpga/rtl /home/bryson/Documents/TENNLab/framework/pyframework/lib/python3.10/site-packages/cocotb/share/lib/verilator/verilator.cpp /home/bryson/Documents/TENNLab/framework/fpga/fpga/rtl/risp_neuron.sv /home/bryson/Documents/TENNLab/framework/fpga/fpga/rtl/risp_synapse.sv /home/bryson/.cache/neuro_fpga/networks/2cde053957.sv /home/bryson/Documents/TENNLab/framework/fpga/fpga/rtl/dispatch_source.sv /home/bryson/Documents/TENNLab/framework/fpga/fpga/rtl/stream_sink.sv /home/bryson/Documents/TENNLab/framework/fpga/fpga/rtl/basic_processor.sv INFO cocotb: Running command: make -C /home/bryson/.cache/neuro_fpga/sims/verilator/test_simple_processor -f Vtop.mk INFO cocotb: make: Entering directory '/home/bryson/.cache/neuro_fpga/sims/verilator/test_simple_processor' INFO cocotb: make: Nothing to be done for 'default'. INFO cocotb: make: Leaving directory '/home/bryson/.cache/neuro_fpga/sims/verilator/test_simple_processor' INFO cocotb: Running command: /home/bryson/.cache/neuro_fpga/sims/verilator/test_simple_processor/basic_processor INFO cocotb: -.--ns INFO gpi ..mbed/gpi_embed.cpp:105 in set_program_name_in_venv Using Python virtual environment interpreter at /home/bryson/Documents/TENNLab/framework/pyframework/bin/python INFO cocotb: -.--ns INFO gpi ../gpi/GpiCommon.cpp:101 in gpi_print_registered_impl VPI registered INFO cocotb: 0.00ns INFO cocotb Running on Verilator version 5.006 2023-01-22 INFO cocotb: 0.00ns INFO cocotb Running tests with cocotb v1.8.1 from /home/bryson/Documents/TENNLab/framework/pyframework/lib/python3.10/site-packages/cocotb INFO cocotb: 0.00ns INFO cocotb Seeding Python random module with 1720467053 INFO cocotb: 0.00ns INFO cocotb.regression Found test test_simple_processor.simple_processor_nominal INFO cocotb: 0.00ns INFO cocotb.regression running simple_processor_nominal (1/1) INFO cocotb: run: 6 INFO cocotb: 0.11ns INFO cocotb.regression simple_processor_nominal failed INFO cocotb: Traceback (most recent call last): INFO cocotb: File "/home/bryson/Documents/TENNLab/framework/fpga/tb/test_simple_processor.py", line 90, in simple_processor_nominal INFO cocotb: await run_processor() INFO cocotb: File "/home/bryson/Documents/TENNLab/framework/fpga/tb/test_simple_processor.py", line 78, in run_processor INFO cocotb: assert dut.out.value == 1 INFO cocotb: AssertionError: assert 0 == 1 INFO cocotb: + where 0 = ModifiableObject(basic_processor.out).value INFO cocotb: + where ModifiableObject(basic_processor.out) = HierarchyObject(basic_processor).out INFO cocotb: 0.11ns INFO cocotb.regression ******************************************************************************************************** INFO cocotb: ** TEST STATUS SIM TIME (ns) REAL TIME (s) RATIO (ns/s) ** INFO cocotb: ******************************************************************************************************** INFO cocotb: ** test_simple_processor.simple_processor_nominal FAIL 0.11 0.01 17.53 ** INFO cocotb: ******************************************************************************************************** INFO cocotb: ** TESTS=1 PASS=0 FAIL=1 SKIP=0 0.11 0.88 0.12 ** INFO cocotb: ******************************************************************************************************** INFO cocotb: INFO cocotb: - :0: Verilog $finish ERROR cocotb: Failed: test_simple_processor::simple_processor_nominal FAILED 1 tests.

[keegandent]

I think I figured out the issue and unfortunately it seems to come back to everyone's favorite: simulator support of language features.

So the inp_val should NOT be 0 during those SPK calls and this isn't a problem for testing live with Vivado, which to me means that it's down to this line in fpga/rtl/dispatch_source.sv

    logic signed [NET_CHARGE_WIDTH-1:0] inp_val
        = src[(`SRC_WIDTH - OPC_WIDTH - $clog2(NET_NUM_INP) - 1) -: NET_CHARGE_WIDTH];

Verilator either doesn't understand mixed declaration and assignment, or it doesn't understand -: syntactic sugar.

I'm wondering if we shouldn't deprecate cocotb honestly. Edalize enables use of SystemVerilog benches in the actual EDA tools users will leverage to target their boards, which may be more useful and appropriate. Conversely, having benches written in Python can be pretty helpful, and we've invested a bit in the testing for it so far. I'm just wondering if it's worth the PITA of making syntax support both simulator AND the EDA tool of choice. Heaven forbid everything just support the entire IEEE language set.

[BGull00]

I will say that I've already spent some time debugging issues with cocotb's Verilator support outside of this issue, so deprecating cocotb in this project would not disappoint me. At the same time, it is nice to have test code you can run just with pytest without any hardware on hand or bulky EDA simulators/verification tools to make sure everything is working as expected. This assumes the python tools and simulator actually work of course though.

[keegandent]

it is nice to have test code you can run just with pytest without any hardware on hand or bulky EDA simulators/verification tools to make sure everything is working as expected.

With edalize this may still be possible. It will run an EDA tool headless or with GUI and require (as far as I know) tests written in HDL instead of Python, but other than that you still get the benefit of it being easy to run your test suites. And even if it has a bug for a particular tool or simulator, a user can still manually create a project with that HDL testbench and get going. As an added bonus, our use of HDL features could only narrow for compatibility with the EDA toolchains for actual FPGA targets as opposed to arbitrary free sims.

This assumes the python tools and simulator actually work of course though.

Yeah this is the hill on which cocotb lives or dies on to me. @gsr4jrobo, are there simulators the EECS department uses widely? Or is it just whatever is included in the ASIC or FPGA design toolchain?

Below are the valid backends for cocotb and edalize respectively. Understandably, edalize is able to support significantly more due to less integration with the tool itself.

[keegandent]

So it was mentioned offline that for ASICs, UTK does use Questa. However, seeing as this is unlikely to be relevant for FPGA development, I don't think it should necessarily play into the decision to keep or drop cocotb.