need to update command on how to run coverage and fix directory path where riscof looks for elf

[dansmathers]

moving issue from https://github.com/riscv-software-src/riscv-config/issues/135

https://riscof.readthedocs.io/en/stable/coverage.html

The current directions say to run coverage: riscof --verbose debug coverage --suite /path/to/suite --env /path/to/suite

this will generate an error. -c or --cgf_file is a required option.

However, even if -cgf_file is used, there is another error. riscof looks for the elf file in the wrong directory:

FileNotFoundError: [Errno 2] No such file or directory: '/riscof/riscof_work/rv32i_m/C/src/caddi-01.S/ref.elf'

but it looks like it should instead be looking in: ./rv32i_m/I/src/addi-01.S/ref/ref.elf

Thanks

I don't think the paths are wrong. Both would probably point to the same thing. All paths are resolved to be absolute paths before the actual execution of riscof starts. The former simply looks like the absolute path of the latter. I think the compilation was not successful. Can you share the entire log?

@dansmathers its better to move/track this issue in the riscof repo since it doesn't have anything to do with riscv-config.

[dansmathers]

riscof coverage --config=config.ini --suite=riscv-arch-test/riscv-test-suite/ --env=riscv-arch-test/riscv-test-suite/env --cgf-file /riscof/riscv-arch-test/coverage/rvi_c.cgf --cgf-file /riscof/riscv-arch-test/coverage/rvi.cgf --cgf-file /riscof/riscv-arch-test/coverage/rvi_fencei.cgf --cgf-file /riscof/riscv-arch-test/coverage/rvi_m.cgf --cgf-file /riscof/riscv-arch-test/coverage/rvi_priv.cgf | tee cov.log INFO | ** RISCOF: RISC-V Architectural Test Framework 1.25.3 * INFO | using riscv_isac version : 0.17.0 INFO | using riscv_config version : 3.9.1 INFO | Reading configuration from: /riscof/config.ini INFO | Preparing Models INFO | Input-ISA file INFO | Loading input file: /riscof/spike/spike_isa.yaml INFO | Load Schema /.local/lib/python3.8/site-packages/riscv_config/schemas/schema_isa.yaml INFO | Processing Hart: hart0 INFO | Initiating Validation INFO | No errors for Hart: 0 :) INFO | Updating fields node for each CSR INFO | Dumping out Normalized Checked YAML: /riscof/riscof_work/spike_isa_checked.yaml INFO | Input-Platform file INFO | Loading input file: /riscof/spike/spike_platform.yaml INFO | Load Schema /.local/lib/python3.8/site-packages/riscv_config/schemas/schema_platform.yaml INFO | Initiating Validation INFO | No Syntax errors in Input Platform Yaml. :) INFO | Dumping out Normalized Checked YAML: /riscof/riscof_work/spike_platform_checked.yaml INFO | Generating database for suite: /riscof/riscv-arch-test-20230726/riscv-test-suite INFO | Database File Generated: /riscof/riscof_work/database.yaml INFO | Env path set to/riscof/riscv-arch-test-20230726/riscv-test-suite/env INFO | Will collect Coverage using RISCV-ISAC INFO | CGF file(s) being used : ('/riscof/riscv-arch-test-20230726/coverage/rvi_c.cgf', '/riscof/riscv-arch-test-20230726/coverage/rvi.cgf', '/riscof/riscv-arch-test-20230726/coverage/rvi_fencei.cgf', '/riscof/riscv-arch-test-20230726/coverage/rvi_m.cgf', '/riscof/riscv-arch-test-20230726/coverage/rvi_priv.cgf') INFO | Selecting Tests. INFO | Running Tests on Reference. ERROR | INFO | ** RISC-V ISA Coverage 0.17.0 * INFO | Copyright (c) 2020, InCore Semiconductors Pvt. Ltd. INFO | All Rights Reserved.

Traceback (most recent call last): File "/.local/bin/riscv_isac", line 8, in sys.exit(cli()) File "/.local/lib/python3.8/site-packages/click/core.py", line 1157, in call return self.main(args, kwargs) File "/.local/lib/python3.8/site-packages/click/core.py", line 1078, in main rv = self.invoke(ctx) File "/.local/lib/python3.8/site-packages/click/core.py", line 1688, in invoke return _process_result(sub_ctx.command.invoke(sub_ctx)) File "/.local/lib/python3.8/site-packages/click/core.py", line 1434, in invoke return ctx.invoke(self.callback, ctx.params) File "/.local/lib/python3.8/site-packages/click/core.py", line 783, in invoke return __callback(args, kwargs) File "/.local/lib/python3.8/site-packages/riscv_isac/main.py", line 136, in coverage isac(output_file,elf,trace_file, window_size, expand_cgf(cgf_file,int(xlen),int(flen),log_redundant), parser_name, decoder_name, parser_path, decoder_path, detailed, test_label, File "/.local/lib/python3.8/site-packages/riscv_isac/cgf_normalize.py", line 565, in expand_cgf cgf = utils.load_cgf(cgf_files) File "/.local/lib/python3.8/site-packages/riscv_isac/utils.py", line 163, in load_cgf safe_yaml.dump(dict(safe_yaml.load(fp)),string_stream) File "/.local/lib/python3.8/site-packages/ruamel/yaml/main.py", line 426, in load return constructor.get_single_data() File "/.local/lib/python3.8/site-packages/ruamel/yaml/constructor.py", line 111, in get_single_data node = self.composer.get_single_node() File "_ruamel_yaml.pyx", line 706, in _ruamel_yaml.CParser.get_single_node File "_ruamel_yaml.pyx", line 724, in _ruamel_yaml.CParser._compose_document File "_ruamel_yaml.pyx", line 775, in _ruamel_yaml.CParser._compose_node File "_ruamel_yaml.pyx", line 889, in _ruamel_yaml.CParser._compose_mapping_node File "_ruamel_yaml.pyx", line 775, in _ruamel_yaml.CParser._compose_node File "_ruamel_yaml.pyx", line 889, in _ruamel_yaml.CParser._compose_mapping_node File "_ruamel_yaml.pyx", line 775, in _ruamel_yaml.CParser._compose_node File "_ruamel_yaml.pyx", line 889, in _ruamel_yaml.CParser._compose_mapping_node File "_ruamel_yaml.pyx", line 743, in _ruamel_yaml.CParser._compose_node ruamel.yaml.composer.ComposerError: found undefined alias in "", line 15, column 11 make: ** [/riscof/riscof_work/Makefile.Reference-sail_c_simulator:5: TARGET0] Error 1 INFO | Merging Coverage reports Traceback (most recent call last): File "/.local/bin/riscof", line 8, in sys.exit(cli()) File "/.local/lib/python3.8/site-packages/click/core.py", line 1157, in call return self.main(args, kwargs) File "/.local/lib/python3.8/site-packages/click/core.py", line 1078, in main rv = self.invoke(ctx) File "/.local/lib/python3.8/site-packages/click/core.py", line 1688, in invoke return _process_result(sub_ctx.command.invoke(sub_ctx)) File "/.local/lib/python3.8/site-packages/click/core.py", line 1434, in invoke return ctx.invoke(self.callback, ctx.params) File "/.local/lib/python3.8/site-packages/click/core.py", line 783, in invoke return __callback(*args, *kwargs) File "/.local/lib/python3.8/site-packages/click/decorators.py", line 33, in new_func return f(get_current_context(), args, kwargs) File "/.local/lib/python3.8/site-packages/riscof/cli.py", line 399, in coverage report, for_html, test_stats, coverpoints = framework.run_coverage(base, isa_file, platform_file, File "/.local/lib/python3.8/site-packages/riscof/framework/main.py", line 126, in run_coverage 'test_size': [str(entry) for entry in find_elf_size(elf)], File "/.local/lib/python3.8/site-packages/riscof/framework/main.py", line 59, in find_elf_size with open(elf, 'rb') as f: FileNotFoundError: [Errno 2] No such file or directory: '/riscof/riscof_work/rv32i_m/C/src/cadd-01.S/ref/ref.elf'

[dansmathers]

@pawks Can you share your command you use to run coverage and I will try that? I was guessing at the correct command because I didn't see a quickstart command that I could just run correctly?

Thanks, Dan

[pawks]

The error you are seeing is because you didnt include the dataset.cgf in the command line. It should be the first file in the list of cgf file arguments. The correct command should look something like the following:

riscof coverage --config=config.ini --suite=riscv-arch-test/riscv-test-suite/ --env=riscv-arch-test/riscv-test-suite/env --cgf-file /riscof/riscv-arch-test/dataset.cgf --cgf-file /riscof/riscv-arch-test/coverage/rvi_c.cgf --cgf-file /riscof/riscv-arch-test/coverage/rvi.cgf --cgf-file /riscof/riscv-arch-test/coverage/rvi_fencei.cgf --cgf-file /riscof/riscv-arch-test/coverage/rvi_m.cgf --cgf-file /riscof/riscv-arch-test/coverage/rvi_priv.cgf

[neelgala]

@dansmathers as we noted in the last SIG call.. the docs for "test contributors" is missing and we will be fixing that asap in the coming weeks. It would be great if you could list out the missing docs you encountered in your struggle so we can accurately address those and others together.

Thanks for the inputs.

[dansmathers]

Thanks. that command helped. I still see errors going by for some tests but the script ran. I was trying to build everything from scratch again to check but it looks like the current github sail build is currently broken. It does seem like it might be hard to do large unified pulls because there are 4 or 5 githubs that need to be kept in sync?