search

riscv-software-src / riscof

BSD 3-Clause "New" or "Revised" License
61 stars 39 forks source link

Some arch tests are failing with riscof #30

Open owaisfarooq-10x opened 2 years ago

owaisfarooq-10x commented 2 years ago

I followed the riscof installation quickstart guide and when I ran the tests using the following command some tests passed and others failed

riscof run --config=config.ini --suite /home/owaisfarooq/riscv-arch-test/riscv-test-suite/rv64i_m --env /home/owaisfarooq/riscv-arch-test/riscv-test-suite/env/

riscof version: RISC-V Architectural Test Framework., version 1.23.2 python3 version: 3.8.10 

    INFO | ****** RISCOF: RISC-V Architectural Test Framework 1.23.2 *******
    INFO | using riscv_isac version : 0.8.0
    INFO | using riscv_config version : 2.10.1
    INFO | Reading configuration from: /home/owaisfarooq/riscof_test/config.ini
    INFO | Preparing Models
    INFO | Input-ISA file
    INFO | Loading input file: /home/owaisfarooq/riscof-plugins/spike_simple/spike_simple_isa.yaml
    INFO | Load Schema /home/owaisfarooq/.local/lib/python3.8/site-packages/riscv_config/schemas/schema_isa.yaml
    INFO | Processing Hart: hart0
    INFO | Initiating Validation
    INFO | No errors for Hart: 0 :)
    INFO | Initiating post processing and reset value checks.
    INFO | Dumping out Normalized Checked YAML: /home/owaisfarooq/riscof_test/riscof_work/spike_simple_isa_checked.yaml
    INFO | Input-Platform file
    INFO | Loading input file: /home/owaisfarooq/riscof-plugins/spike_simple/spike_simple_platform.yaml
    INFO | Load Schema /home/owaisfarooq/.local/lib/python3.8/site-packages/riscv_config/schemas/schema_platform.yaml
    INFO | Initiating Validation
    INFO | No Syntax errors in Input Platform Yaml. :)
    INFO | Dumping out Normalized Checked YAML: /home/owaisfarooq/riscof_test/riscof_work/spike_simple_platform_checked.yaml
    INFO | Generating database for suite: /home/owaisfarooq/riscv-arch-test/riscv-test-suite/rv64i_m
    INFO | Database File Generated: /home/owaisfarooq/riscof_test/riscof_work/database.yaml
    INFO | Env path set to/home/owaisfarooq/riscv-arch-test/riscv-test-suite/env
    INFO | Running Build for DUT
    INFO | Running Build for Reference
    INFO | Selecting Tests.
    INFO | Running Tests on DUT.
    INFO | Running Tests on Reference Model.
    INFO | Initiating signature checking.
    INFO | Following 116 tests have been run :

    INFO | TEST NAME                                          : COMMIT ID                                : STATUS
   ERROR | /home/owaisfarooq/riscv-arch-test/riscv-test-suite/rv64i_m/C/src/cadd-01.S : -                                        : Failed
    INFO | /home/owaisfarooq/riscv-arch-test/riscv-test-suite/rv64i_m/C/src/caddi-01.S : -                                        : Passed
    INFO | /home/owaisfarooq/riscv-arch-test/riscv-test-suite/rv64i_m/C/src/caddi16sp-01.S : -                                        : Passed
   ERROR | /home/owaisfarooq/riscv-arch-test/riscv-test-suite/rv64i_m/C/src/caddi4spn-01.S : -                                        : Failed
    INFO | /home/owaisfarooq/riscv-arch-test/riscv-test-suite/rv64i_m/C/src/caddiw-01.S : -                                        : Passed
    INFO | /home/owaisfarooq/riscv-arch-test/riscv-test-suite/rv64i_m/C/src/caddw-01.S : -                                        : Passed
   ERROR | /home/owaisfarooq/riscv-arch-test/riscv-test-suite/rv64i_m/C/src/cand-01.S : -                                        : Failed
   ERROR | /home/owaisfarooq/riscv-arch-test/riscv-test-suite/rv64i_m/C/src/candi-01.S : -                                        : Failed
   ERROR | /home/owaisfarooq/riscv-arch-test/riscv-test-suite/rv64i_m/C/src/cbeqz-01.S : -                                        : Failed
   ERROR | /home/owaisfarooq/riscv-arch-test/riscv-test-suite/rv64i_m/C/src/cbnez-01.S : -                                        : Failed
   ERROR | /home/owaisfarooq/riscv-arch-test/riscv-test-suite/rv64i_m/C/src/cebreak-01.S : -                                        : Failed
    INFO | /home/owaisfarooq/riscv-arch-test/riscv-test-suite/rv64i_m/C/src/cj-01.S : -                                        : Passed
   ERROR | /home/owaisfarooq/riscv-arch-test/riscv-test-suite/rv64i_m/C/src/cjalr-01.S : -                                        : Failed
   ERROR | /home/owaisfarooq/riscv-arch-test/riscv-test-suite/rv64i_m/C/src/cjr-01.S : -                                        : Failed
   ERROR | /home/owaisfarooq/riscv-arch-test/riscv-test-suite/rv64i_m/C/src/cld-01.S : -                                        : Failed
   ERROR | /home/owaisfarooq/riscv-arch-test/riscv-test-suite/rv64i_m/C/src/cldsp-01.S : -                                        : Failed
    INFO | /home/owaisfarooq/riscv-arch-test/riscv-test-suite/rv64i_m/C/src/cli-01.S : -                                        : Passed
   ERROR | /home/owaisfarooq/riscv-arch-test/riscv-test-suite/rv64i_m/C/src/clui-01.S : -                                        : Failed
   ERROR | /home/owaisfarooq/riscv-arch-test/riscv-test-suite/rv64i_m/C/src/clw-01.S : -                                        : Failed
   ERROR | /home/owaisfarooq/riscv-arch-test/riscv-test-suite/rv64i_m/C/src/clwsp-01.S : -                                        : Failed
   ERROR | /home/owaisfarooq/riscv-arch-test/riscv-test-suite/rv64i_m/C/src/cmv-01.S : -                                        : Failed
    INFO | /home/owaisfarooq/riscv-arch-test/riscv-test-suite/rv64i_m/C/src/cnop-01.S : -                                        : Passed
    INFO | /home/owaisfarooq/riscv-arch-test/riscv-test-suite/rv64i_m/C/src/cor-01.S : -                                        : Passed
    INFO | /home/owaisfarooq/riscv-arch-test/riscv-test-suite/rv64i_m/C/src/csd-01.S : -                                        : Passed
   ERROR | /home/owaisfarooq/riscv-arch-test/riscv-test-suite/rv64i_m/C/src/csdsp-01.S : -                                        : Failed
    INFO | /home/owaisfarooq/riscv-arch-test/riscv-test-suite/rv64i_m/C/src/cslli-01.S : -                                        : Passed
    INFO | /home/owaisfarooq/riscv-arch-test/riscv-test-suite/rv64i_m/C/src/csrai-01.S : -                                        : Passed
   ERROR | /home/owaisfarooq/riscv-arch-test/riscv-test-suite/rv64i_m/C/src/csrli-01.S : -                                        : Failed
   ERROR | /home/owaisfarooq/riscv-arch-test/riscv-test-suite/rv64i_m/C/src/csub-01.S : -                                        : Failed
    INFO | /home/owaisfarooq/riscv-arch-test/riscv-test-suite/rv64i_m/C/src/csubw-01.S : -                                        : Passed
   ERROR | /home/owaisfarooq/riscv-arch-test/riscv-test-suite/rv64i_m/C/src/csw-01.S : -                                        : Failed
   ERROR | /home/owaisfarooq/riscv-arch-test/riscv-test-suite/rv64i_m/C/src/cswsp-01.S : -                                        : Failed
   ERROR | /home/owaisfarooq/riscv-arch-test/riscv-test-suite/rv64i_m/C/src/cxor-01.S : -                                        : Failed
   ERROR | /home/owaisfarooq/riscv-arch-test/riscv-test-suite/rv64i_m/I/src/add-01.S : -                                        : Failed
   ERROR | /home/owaisfarooq/riscv-arch-test/riscv-test-suite/rv64i_m/I/src/addi-01.S : -                                        : Failed
   ERROR | /home/owaisfarooq/riscv-arch-test/riscv-test-suite/rv64i_m/I/src/addiw-01.S : -                                        : Failed
    INFO | /home/owaisfarooq/riscv-arch-test/riscv-test-suite/rv64i_m/I/src/addw-01.S : -                                        : Passed
   ERROR | /home/owaisfarooq/riscv-arch-test/riscv-test-suite/rv64i_m/I/src/and-01.S : -                                        : Failed
   ERROR | /home/owaisfarooq/riscv-arch-test/riscv-test-suite/rv64i_m/I/src/andi-01.S : -                                        : Failed
   ERROR | /home/owaisfarooq/riscv-arch-test/riscv-test-suite/rv64i_m/I/src/auipc-01.S : -                                        : Failed
   ERROR | /home/owaisfarooq/riscv-arch-test/riscv-test-suite/rv64i_m/I/src/beq-01.S : -                                        : Failed
    INFO | /home/owaisfarooq/riscv-arch-test/riscv-test-suite/rv64i_m/I/src/bge-01.S : -                                        : Passed
   ERROR | /home/owaisfarooq/riscv-arch-test/riscv-test-suite/rv64i_m/I/src/bgeu-01.S : -                                        : Failed
    INFO | /home/owaisfarooq/riscv-arch-test/riscv-test-suite/rv64i_m/I/src/blt-01.S : -                                        : Passed
    INFO | /home/owaisfarooq/riscv-arch-test/riscv-test-suite/rv64i_m/I/src/bltu-01.S : -                                        : Passed
    INFO | /home/owaisfarooq/riscv-arch-test/riscv-test-suite/rv64i_m/I/src/bne-01.S : -                                        : Passed
   ERROR | /home/owaisfarooq/riscv-arch-test/riscv-test-suite/rv64i_m/I/src/fence-01.S : -                                        : Failed
    INFO | /home/owaisfarooq/riscv-arch-test/riscv-test-suite/rv64i_m/I/src/jal-01.S : -                                        : Passed
   ERROR | /home/owaisfarooq/riscv-arch-test/riscv-test-suite/rv64i_m/I/src/jalr-01.S : -                                        : Failed
    INFO | /home/owaisfarooq/riscv-arch-test/riscv-test-suite/rv64i_m/I/src/lb-align-01.S : -                                        : Passed
   ERROR | /home/owaisfarooq/riscv-arch-test/riscv-test-suite/rv64i_m/I/src/lbu-align-01.S : -                                        : Failed
    INFO | /home/owaisfarooq/riscv-arch-test/riscv-test-suite/rv64i_m/I/src/ld-align-01.S : -                                        : Passed
    INFO | /home/owaisfarooq/riscv-arch-test/riscv-test-suite/rv64i_m/I/src/lh-align-01.S : -                                        : Passed
    INFO | /home/owaisfarooq/riscv-arch-test/riscv-test-suite/rv64i_m/I/src/lhu-align-01.S : -                                        : Passed
   ERROR | /home/owaisfarooq/riscv-arch-test/riscv-test-suite/rv64i_m/I/src/lui-01.S : -                                        : Failed
    INFO | /home/owaisfarooq/riscv-arch-test/riscv-test-suite/rv64i_m/I/src/lw-align-01.S : -                                        : Passed
    INFO | /home/owaisfarooq/riscv-arch-test/riscv-test-suite/rv64i_m/I/src/lwu-align-01.S : -                                        : Passed
    INFO | /home/owaisfarooq/riscv-arch-test/riscv-test-suite/rv64i_m/I/src/or-01.S : -                                        : Passed
    INFO | /home/owaisfarooq/riscv-arch-test/riscv-test-suite/rv64i_m/I/src/ori-01.S : -                                        : Passed
   ERROR | /home/owaisfarooq/riscv-arch-test/riscv-test-suite/rv64i_m/I/src/sb-align-01.S : -                                        : Failed
    INFO | /home/owaisfarooq/riscv-arch-test/riscv-test-suite/rv64i_m/I/src/sd-align-01.S : -                                        : Passed
   ERROR | /home/owaisfarooq/riscv-arch-test/riscv-test-suite/rv64i_m/I/src/sh-align-01.S : -                                        : Failed
   ERROR | /home/owaisfarooq/riscv-arch-test/riscv-test-suite/rv64i_m/I/src/sll-01.S : -                                        : Failed
   ERROR | /home/owaisfarooq/riscv-arch-test/riscv-test-suite/rv64i_m/I/src/slli-01.S : -                                        : Failed
    INFO | /home/owaisfarooq/riscv-arch-test/riscv-test-suite/rv64i_m/I/src/slliw-01.S : -                                        : Passed
   ERROR | /home/owaisfarooq/riscv-arch-test/riscv-test-suite/rv64i_m/I/src/sllw-01.S : -                                        : Failed
    INFO | /home/owaisfarooq/riscv-arch-test/riscv-test-suite/rv64i_m/I/src/slt-01.S : -                                        : Passed
   ERROR | /home/owaisfarooq/riscv-arch-test/riscv-test-suite/rv64i_m/I/src/slti-01.S : -                                        : Failed
   ERROR | /home/owaisfarooq/riscv-arch-test/riscv-test-suite/rv64i_m/I/src/sltiu-01.S : -                                        : Failed
   ERROR | /home/owaisfarooq/riscv-arch-test/riscv-test-suite/rv64i_m/I/src/sltu-01.S : -                                        : Failed
    INFO | /home/owaisfarooq/riscv-arch-test/riscv-test-suite/rv64i_m/I/src/sra-01.S : -                                        : Passed
    INFO | /home/owaisfarooq/riscv-arch-test/riscv-test-suite/rv64i_m/I/src/srai-01.S : -                                        : Passed
    INFO | /home/owaisfarooq/riscv-arch-test/riscv-test-suite/rv64i_m/I/src/sraiw-01.S : -                                        : Passed
    INFO | /home/owaisfarooq/riscv-arch-test/riscv-test-suite/rv64i_m/I/src/sraw-01.S : -                                        : Passed
   ERROR | /home/owaisfarooq/riscv-arch-test/riscv-test-suite/rv64i_m/I/src/srl-01.S : -                                        : Failed
   ERROR | /home/owaisfarooq/riscv-arch-test/riscv-test-suite/rv64i_m/I/src/srli-01.S : -                                        : Failed
    INFO | /home/owaisfarooq/riscv-arch-test/riscv-test-suite/rv64i_m/I/src/srliw-01.S : -                                        : Passed
    INFO | /home/owaisfarooq/riscv-arch-test/riscv-test-suite/rv64i_m/I/src/srlw-01.S : -                                        : Passed
    INFO | /home/owaisfarooq/riscv-arch-test/riscv-test-suite/rv64i_m/I/src/sub-01.S : -                                        : Passed
    INFO | /home/owaisfarooq/riscv-arch-test/riscv-test-suite/rv64i_m/I/src/subw-01.S : -                                        : Passed
    INFO | /home/owaisfarooq/riscv-arch-test/riscv-test-suite/rv64i_m/I/src/sw-align-01.S : -                                        : Passed
   ERROR | /home/owaisfarooq/riscv-arch-test/riscv-test-suite/rv64i_m/I/src/xor-01.S : -                                        : Failed
    INFO | /home/owaisfarooq/riscv-arch-test/riscv-test-suite/rv64i_m/I/src/xori-01.S : -                                        : Passed
    INFO | /home/owaisfarooq/riscv-arch-test/riscv-test-suite/rv64i_m/M/src/div-01.S : -                                        : Passed
    INFO | /home/owaisfarooq/riscv-arch-test/riscv-test-suite/rv64i_m/M/src/divu-01.S : -                                        : Passed
   ERROR | /home/owaisfarooq/riscv-arch-test/riscv-test-suite/rv64i_m/M/src/divuw-01.S : -                                        : Failed
   ERROR | /home/owaisfarooq/riscv-arch-test/riscv-test-suite/rv64i_m/M/src/divw-01.S : -                                        : Failed
    INFO | /home/owaisfarooq/riscv-arch-test/riscv-test-suite/rv64i_m/M/src/mul-01.S : -                                        : Passed
   ERROR | /home/owaisfarooq/riscv-arch-test/riscv-test-suite/rv64i_m/M/src/mulh-01.S : -                                        : Failed
   ERROR | /home/owaisfarooq/riscv-arch-test/riscv-test-suite/rv64i_m/M/src/mulhsu-01.S : -                                        : Failed
    INFO | /home/owaisfarooq/riscv-arch-test/riscv-test-suite/rv64i_m/M/src/mulhu-01.S : -                                        : Passed
    INFO | /home/owaisfarooq/riscv-arch-test/riscv-test-suite/rv64i_m/M/src/mulw-01.S : -                                        : Passed
    INFO | /home/owaisfarooq/riscv-arch-test/riscv-test-suite/rv64i_m/M/src/rem-01.S : -                                        : Passed
    INFO | /home/owaisfarooq/riscv-arch-test/riscv-test-suite/rv64i_m/M/src/remu-01.S : -                                        : Passed
    INFO | /home/owaisfarooq/riscv-arch-test/riscv-test-suite/rv64i_m/M/src/remuw-01.S : -                                        : Passed
    INFO | /home/owaisfarooq/riscv-arch-test/riscv-test-suite/rv64i_m/M/src/remw-01.S : -                                        : Passed
    INFO | /home/owaisfarooq/riscv-arch-test/riscv-test-suite/rv64i_m/Zifencei/src/Fencei.S : -                                        : Passed
   ERROR | /home/owaisfarooq/riscv-arch-test/riscv-test-suite/rv64i_m/privilege/src/ebreak.S : -                                        : Failed
   ERROR | /home/owaisfarooq/riscv-arch-test/riscv-test-suite/rv64i_m/privilege/src/ecall.S : -                                        : Failed
   ERROR | /home/owaisfarooq/riscv-arch-test/riscv-test-suite/rv64i_m/privilege/src/misalign-beq-01.S : -                                        : Failed
   ERROR | /home/owaisfarooq/riscv-arch-test/riscv-test-suite/rv64i_m/privilege/src/misalign-bge-01.S : -                                        : Failed
   ERROR | /home/owaisfarooq/riscv-arch-test/riscv-test-suite/rv64i_m/privilege/src/misalign-bgeu-01.S : -                                        : Failed
   ERROR | /home/owaisfarooq/riscv-arch-test/riscv-test-suite/rv64i_m/privilege/src/misalign-blt-01.S : -                                        : Failed
   ERROR | /home/owaisfarooq/riscv-arch-test/riscv-test-suite/rv64i_m/privilege/src/misalign-bltu-01.S : -                                        : Failed
   ERROR | /home/owaisfarooq/riscv-arch-test/riscv-test-suite/rv64i_m/privilege/src/misalign-bne-01.S : -                                        : Failed
   ERROR | /home/owaisfarooq/riscv-arch-test/riscv-test-suite/rv64i_m/privilege/src/misalign-jal-01.S : -                                        : Failed
   ERROR | /home/owaisfarooq/riscv-arch-test/riscv-test-suite/rv64i_m/privilege/src/misalign-ld-01.S : -                                        : Failed
   ERROR | /home/owaisfarooq/riscv-arch-test/riscv-test-suite/rv64i_m/privilege/src/misalign-lh-01.S : -                                        : Failed
   ERROR | /home/owaisfarooq/riscv-arch-test/riscv-test-suite/rv64i_m/privilege/src/misalign-lhu-01.S : -                                        : Failed
   ERROR | /home/owaisfarooq/riscv-arch-test/riscv-test-suite/rv64i_m/privilege/src/misalign-lw-01.S : -                                        : Failed
   ERROR | /home/owaisfarooq/riscv-arch-test/riscv-test-suite/rv64i_m/privilege/src/misalign-lwu-01.S : -                                        : Failed
   ERROR | /home/owaisfarooq/riscv-arch-test/riscv-test-suite/rv64i_m/privilege/src/misalign-sd-01.S : -                                        : Failed
   ERROR | /home/owaisfarooq/riscv-arch-test/riscv-test-suite/rv64i_m/privilege/src/misalign-sh-01.S : -                                        : Failed
   ERROR | /home/owaisfarooq/riscv-arch-test/riscv-test-suite/rv64i_m/privilege/src/misalign-sw-01.S : -                                        : Failed
   ERROR | /home/owaisfarooq/riscv-arch-test/riscv-test-suite/rv64i_m/privilege/src/misalign1-jalr-01.S : -                                        : Failed
   ERROR | /home/owaisfarooq/riscv-arch-test/riscv-test-suite/rv64i_m/privilege/src/misalign2-jalr-01.S : -                                        : Failed
    INFO | Test report generated at /home/owaisfarooq/riscof_test/riscof_work/report.html.
    INFO | Opening test report in web-browser
neelgala commented 2 years ago

couple of questions:

  1. What version of spike and sail are you using? Are they the latest versions?
  2. can you share the report.html file here ?
pawks commented 2 years ago

@owaisfarooq-10x I tried to recreate this with the latest version of riscv-toolchain, SPIKE, SAIL and riscof on a fresh install. All the tests pass. Please check the versions of the tools you are using and share the report.html file.

owaisfarooq-10x commented 2 years ago

I used the latest version of every tool. I also used the latest sail plugins available at riscof-plugins repo. The issue is with riscof-plugins for sail.