run.py fails to check return codes for verilog compile and generation

run.py --steps_gen fails to check the return codes correctly in Questa and Xcelium for Verilog compile errors : see log excerpt below for Cadence-Xcelium where there is a verilog compile error for user_extension/riscv_core_setting.sv but run.py carries on to run the generator despite this message: xrun: *E,VLGERR: An error occurred during parsing. Review the log file for errors with the code *E and fix those identified problems to proceed. Exiting with code (status 1).

I see the same for Random generation errors. This makes it difficult for any tooling to properly detect failures and report them as test compile errors before an RTL simulation is attempted

Example of verilog error for Cadence-Xcelium

Wed, 13 Nov 2019 16:12:55 run.py:173   DEBUG    Compile command: xrun -64bit -access +rwc -f tools/riscv-dv/files.f +incdir+tools/riscv-dv/user_extension +incdir+tools/riscv-dv/user_extension -xmlibdirname sim-riscv-dv/build/xcelium.d -q -sv -uvm -vlog_ext +.vh -I. -uvmhome CDNS-1.2 -elaborate -l sim-riscv-dv/build/compile.log 
Wed, 13 Nov 2019 16:12:55 lib.py:111   DEBUG    xrun -64bit -access +rwc -f tools/riscv-dv/files.f +incdir+tools/riscv-dv/user_extension +incdir+tools/riscv-dv/user_extension -xmlibdirname sim-riscv-dv/build/xcelium.d -q -sv -uvm -vlog_ext +.vh -I. -uvmhome CDNS-1.2 -elaborate -l sim-riscv-dv/build/compile.log 
Wed, 13 Nov 2019 16:12:57 lib.py:127   DEBUG                              };
                          |
xmvlog: *E,ILLPRI (tools/riscv-dv/user_extension/riscv_core_setting.sv,113|11): illegal expression primary [4.2(IEEE)].
(`include file: tools/riscv-dv/user_extension/riscv_core_setting.sv line 113, file: ./src/riscv_instr_pkg.sv line 715)
  import riscv_instr_pkg::*;
                       |
xmvlog: *E,NOPBIND (./test/riscv_instr_test_pkg.sv,20|23): Package riscv_instr_pkg could not be bound.
(`include file: ./test/riscv_instr_base_test.sv line 19, file: ./test/riscv_instr_test_pkg.sv line 22)
  riscv_instr_gen_config  cfg;
.....
....
                           |
xmvlog: *E,NOPBIND (./test/riscv_instr_gen_tb_top.sv,20|28): Package riscv_instr_test_pkg could not be bound.
**xrun: *E,VLGERR: An error occurred during parsing.  Review the log file for errors with the code *E and fix those identified problems to proceed.  Exiting with code (status 1).

Wed, 13 Nov 2019 16:12:57 run.py:185   INFO     Running RISC-V instruction generator**
Wed, 13 Nov 2019 16:12:57 run.py:189   INFO     Generating 1 riscv_arithmetic_basic_test
Wed, 13 Nov 2019 16:12:57 run.py:211   INFO     Running riscv_arithmetic_basic_test with 1 batches
Wed, 13 Nov 2019 16:12:57 run.py:237   INFO     Running riscv_arithmetic_basic_test, batch 1/1, test_cnt:1
Wed, 13 Nov 2019 16:12:57 lib.py:111   DEBUG     xrun -R +uvm_set_type_override=riscv_asm_program_gen,aril_asm_program_gen -svseed 62981450 -xmlibdirnamesim-riscv-dv/build/xcelium.d +UVM_TESTNAME=riscv_instr_base_test  +num_of_tests=1  +start_idx=0  +asm_file_name=sim-riscv-dv/build/asm_tests/riscv_arithmetic_basic_test  -l sim-riscv-dv/build/sim_riscv_arithmetic_basic_test_0.log +instr_cnt=10000 +num_of_sub_program=0 +no_fence=1 +no_data_page=1 +no_branch_jump=1 +boot_mode=m +no_csr_instr=1

Wed, 13 Nov 2019 16:12:57 lib.py:127   DEBUG    TOOL:   xrun(64)    19.03-s013: Started on Nov 13, 2019 at 16:12:57 PST
xrun(64): 19.03-s013: (c) Copyright 1995-2019 Cadence Design Systems, Inc.
xrun: *E,RFAIL: A previous invocation failed while running the subexec xmvlog.  -R/-r cannot be used until the design has been successfully compiled and elaborated.
xrun: *E,NOSTUP: A problem was detected in the setup for simulation. Simulation can be done only after successfully completing design file parsing and elaboration. Also check the command line for any unintentional errors, like omission of the -name or the -xmlibdirname option, if it was used for parsing, earlier.
TOOL:   xrun(64)    19.03-s013: Exiting on Nov 13, 2019 at 16:12:57 PST  (total: 00:00:00)

chipsalliance / riscv-dv

run.py fails to check return codes for verilog compile and generation #305