Are bitmanip extensions supported and how?

[algrobman]

Are ZB bitmanip extensions supported? If yes, how do I specify supported ZB extensions in isa.yaml

[stnolting]

Yes they are:

ISA: RV32IMCUZicsr_Zifencei_Zba_Zbb_Zbc_Zbs

For example, these ISA extensions are used by this port of RISCOF: https://github.com/stnolting/neorv32-riscof/blob/6aba719d8fe7f0ff6bf80743b0b76aa55ef8b098/plugin-neorv32/neorv32_isa.yaml#L4

[algrobman]

Very interesting :

if my ISA : ISA: RV32IMCZicsr_Zifencei_Zba

Everything passing, including 3 ZBA tests.

if it is ISA: RV32IMCZicsr_Zifencei_Zba_Zbb

Getting following

ESC[32m    INFOESC[0m | ESC[32mInitiating signature checking.ESC[0m
ESC[1;31m   ERRORESC[0m | ESC[1;31mSignature file : riscof_work/rv32i_m/M/src/rem-01.S/dut/DUT-s200.signature does not existESC[0m

seems there is 300s timeout for running DUT simulations:

ESC[1;31m   ERRORESC[0m | ESC[1;31mProcess Killed.ESC[0m
ESC[1;31m   ERRORESC[0m | ESC[1;31mCommand did not exit within 300 seconds: Command 'make -k -j1 -f riscof_work/Makefile.DUT-s200 TARGET0 TARGET1 TARGET2 TARGET3 TARGET4 TARGET5 TARGET6 TARGET7 TARGET8 TARGET9 TARGET10 TARGET11 TARGET12 TARGET13 TARGET14 TARGET15 TARGET16 TARGET17 TARGET18 TARGET19 TARGET20 TARGET21 TARGET22 TARGET23 TARGET24 TARGET25 TARGET26 TARGET27 TARGET28 TARGET29 TARGET30 TARGET31 TARGET32 TARGET33 TARGET34 TARGET35 TARGET36 TARGET37 TARGET38 TARGET39 TARGET40 TARGET41 TARGET42 TARGET43 TARGET44 TARGET45 TARGET46 TARGET47 TARGET48 TARGET49 TARGET50 TARGET51 TARGET52 TARGET53 TARGET54 TARGET55 TARGET56 TARGET57 TARGET58 TARGET59 TARGET60 TARGET61 TARGET62 TARGET63 TARGET64 TARGET65 TARGET66 TARGET67 TARGET68 TARGET69 TARGET70 TARGET71 TARGET72 TARGET73 TARGET74 TARGET75 TARGET76 TARGET77 TARGET78 TARGET79 TARGET80 TARGET81 TARGET82 TARGET83 TARGET84 TARGET85 TARGET86 TARGET87 TARGET88 TARGET89 TARGET90 TARGET91 TARGET92 TARGET93 TARGET94 TARGET95 TARGET96 TARGET97 TARGET98 TARGET99 TARGET100 TARGET101 TARGET102 TARGET103 TARGET104 TARGET105 TARGET106 TARGET107 TARGET108 TARGET109 TARGET110' **timed out after 300 secondsES**C[0m

Since I'm running verilog the simulations can take more time than current 300s (and I 'm running one of the smallest CPU models - some will simulate much longer ...)

how do I increase the timeout?

[neelgala]

when you are using make.execute_all() in your model (example here) you can provide the timeout=<val> argument. See below:

make.execute_all(self.work_dir, timeout= 500)

[algrobman]

@neelgala, riscof behavior after timeout seems not reasonable - instead of terminating the run with Timeout message, riscof continue to signature comparison stage and fails there with misleading message of missing signature file .

BTW, why does the system run all tests on DUT, then on the all tests on reference and then compare signature, compiling the same test twice? More reasonable way would be compile a test, run it on DUT and reference, compare result - print comparison status - such way operator shouldn't wait for all tests simulation to see an issue with a test or system setup.

[pawks]

@neelgala, riscof behavior after timeout seems not reasonable - instead of terminating the run with Timeout message, riscof continue to signature comparison stage and fails there with misleading message of missing signature file .

This is primarily controlled by the plugins. The makefile utility does return the return code of the program to the plugin. You can use this to catch this scenario and terminate.

BTW, why does the system run all tests on DUT, then on the all tests on reference and then compare signature,

This was a conscious design choice to enable highly parallel execution while still keeping the framework lean. Especially considering the fact that each model/RTL requires its own environment with a diverse set of requirements.

compiling the same test twice?

The tests need to be compiled twice because the same test cannot be expected to run on both the model and the RTL. The RTL is free to implement custom routines for all of the following and the same may or may not be compliant with the specifications of the model.

• Boot
• Halt (including mechanism for signature dumping)
• Raising Software interrupts
• Handling Timer interrupts
• CLINT etc.

These routines may also contain custom instructions which aren't implemented in the model. Another factor to consider here are the PMAs of each implementation. The tests are crafted to execute in a way where the PMA of any memory region does not affect the outcome of the test, but the model is not equipped to handle any arbitrary PMA configuration.

More reasonable way would be compile a test, run it on DUT and reference, compare result - print comparison status - such way operator shouldn't wait for all tests simulation to see an issue with a test or system setup.

This is definitely a good feature. Do you have any suggestions on how to implement this without placing restrictions on how the users execute the tests on their RTL?

[algrobman]

The tests need to be compiled twice because the same test cannot be expected to run on both the model and the RTL. The RTL is free to implement custom routines for all of the following and the same may or may not be compliant with the specifications of the model.

this defeats the purpose of compliance testing meaning that reference and DUT execute different code.

This was a conscious design choice to enable highly parallel execution while still keeping the framework lean. Especially considering the fact that each model/RTL requires its own environment with a diverse set of requirements.

this parallel execution is possible with emulator targets only, which is not user common case, where the target is or real silicon/evaluation board or FPGA board or RTL. there is no sense to run simultaneous simulations with make on the same CPU server. It would be much more convinient to switch riscof default behavior to sequential test execution /comparison, especially when just debugging modifications done after riscof setup.

BTW there is a lot of redundancy in the files naming - or use separate directories (dut/ref) or use one directory per test with ref/dut suffix or prefix in file names ..

This is definitely a good feature. Do you have any suggestions on how to implement this without placing restrictions on how the users execute the tests on their RTL?

what kind of restriction the sequential execution does place? Even now with "multi target" makefile I had to execute the RTL simulations sequentially because of the server capacity. (and it does not take too much time: less than 10min. I don't think somebody will use riscof/compliance tests for day to day regressions. These tests are too weak and the riscof is not suitable for automatic regressions.