how to change the start address of cva6

[dvusingh]

Hi,

I am doing the verification of cva6. I want to know how start address is generating or assigned as per the linkers script inside verif/tests/. May I know how to modify or change the start address.

I found same issue mentioned in https://github.com/openhwgroup/cva6/issues/1894. How can I modify the start address (can you refer related files)?

Thanks.

[valentinThomazic]

You have to change it in the linker script used. It depends on the test, for instance you can see the one used for the hello world test on the README of the repo ('Running standalone simulations' section)

[JeanRochCoulon]

The testbench setup the boot_address input port of the cva6. You have to change it.

[JeanRochCoulon]

If you need to cooperate to verify cva6, there is room to collaborate. Please contact me.

[dvusingh]

The testbench setup the boot_address input port of the cva6. You have to change it.

I tried to change every address of boot_addr , but still the error is coming.

Fri, 08 Mar 2024 02:35:19 ERROR ERROR return code: True/2, cmd:make veri-testharness target=cv32a6_imac_sv32 variant=rv32imac_zicsr_zifencei elf=/home/new_cva6/cva6/verif/sim/out_2024-03-08/directed_c_tests/median_main.o path_var=/home/new_cva6/cva6/ tool_path=/home/new_cva6/cva6/tools/spike/bin isscomp_opts="" issrun_opts="+debug_disable=1 +ntb_random_seed=1" isspostrun_opts="0x0000000080000000" log=/home/new_cva6/cva6/verif/sim/out_2024-03-08/veri-testharness_sim/median_main.log &> /home/new_cva6/cva6/verif/sim/out_2024-03-08/veri-testharness_sim/median_main.log.iss

Thanks.

[JeanRochCoulon]

I gave you some inputs, but of course you have to debug/investigate/fine tune it. And it would be great to feedback if you succeed. Good luck, hoping hear your solution soon.

[dvusingh]

Thanks!

[dvusingh]

Hi,

I am running the median_test with below target. DV_SIMULATOR: veri_testharness. DV_TARGET: cv32a6_imac_sv32

I changed the boot_addr =0x0000_0000 in the blew mentioned files:

Linker: verif/tests/custom/common/test.ld core/include/cv32a6_imac_sv32_config_pkg.sv core/cva6.sv

The start address is changing but getting the below error message from trace_rvfi_hart_00.dasm file: ILLEGAL_INSTR exception @ 0x0000000000000000

2. After this step, I also modifed the below files, but it doesn't work.

core/frontend/frontend.sv corev_apu/tb/ariane_testharness.sv [corev_apu/src/ariane.sv] [verif/tb/uvmt/uvmt_cva6_dut_wrap.sv] [verif/env/uvme/uvma_cva6_core_cntrl_drv.sv] [verif/env/uvme/uvme_cva6_cfg.sv] [verif/tb/uvmt/cva6_tb_wrapper.sv] [corev_apu/fpga/src/ariane_xilinx.sv] [core/csr_regfile.sv] [corev_apu/openpiton/ariane_verilog_wrap.sv]

Can you suggest any file that I am missing? This same file can be used for other simulators also?

Thanks

[SShanmukh-cell]

Hi @JeanRochCoulon I have tried running the benchmark with vcs-uvm simulator, the test is passing from vcs-uvm (by changing below mentioned files). But Spike is still throwing errors. I have changed the following files:

1. verif/env/uvme/uvme_cva6_cfg.sv
2. core/include/cv32a6_imac_sv32_config_pkg.sv
3. verif/tests/custom/common/test.ld
4. verif/core-v-verif/lib/uvm_agents/uvma_rvfi/uvma_rvfi_utils.sv
5. verif/core-v-verif/vendor/riscv/riscv-isa-sim_old/riscv/Proc.cc

But spike is giving the below error: Access exception occurred while loading payload $home/new_cva6/cva6/verif/sim/out_2024-03-25/directed_c_tests/median_main.o: Memory address 0x0 is invalid

@JeanRochCoulon , is there any other file that i am missing? Can you please help?

[JeanRochCoulon]

I think that Spike inline options need to be fine-tuned to indicate the new start address.

[SShanmukh-cell]

Please, could you explain the process to me?

[SShanmukh-cell]

I have changed following files:

1. verif/core-v-verif/vendor/riscv/riscv-isa-sim_old/arch_test_target/spike/link.ld
2. verif/core-v-verif/vendor/riscv/riscv-isa-sim/arch_test_target/spike/link.ld
3. verif/core-v-verif/vendor/riscv/riscv-isa-sim_old/spike_main/spike.cc
4. verif/core-v-verif/vendor/riscv/riscv-isa-sim_old/riscv/Simulation.cc

I have also added --pc in the Makefile spike: LD_LIBRARY_PATH="$$(realpath ../../tools/spike/lib):$$LD_LIBRARY_PATH" \ $(tool_path)/spike $(spike_stepout) $(spike_extension) --log-commits --isa=$(variant) --priv=$(priv) -l $(elf) --pc=0x10000000 cp $(log).iss $(log)

But still spike is throwing this error(if i give start address as 0x00000000): Access exception occurred while loading payload $home/new_cva6/cva6/verif/sim/out_2024-03-28/directed_c_tests/median_main.o: Memory address 0x0 is invalid

and, if I give start address as 0x10000000, than the error message is : Access exception occurred while loading payload $home/new_cva6/cva6/verif/sim/out_2024-03-28/directed_c_tests/median_main.o: Memory address 0x10000150 is invalid

Can you please help me with this, as I couldn't able to find the solution?

[SShanmukh-cell]

Hi @JeanRochCoulon, can you please help me with the spike? I still couldn't able to figure out where(in which files) and how to give the start address in Spike.

[JeanRochCoulon]

@zchamski, @SShanmukh-cell and us (in cv32a65x configuration) are trying to start execution at 0x0. Can you check whether Spike is able to be used in such a way ? @SShanmukh-cell Could you provide us the modifications you made in RTL ?

[SShanmukh-cell]

Spike is starting from the address 0x10000 and after 5-6 instructions it's jumping to start address 0x8000_0000, which is start address defined in linker file and verif/env/uvme/uvme_cva6_cfg.sv file(Note: I am using cv32_imac_sv32_config_pkg ).

spike log(in case of start address 0x8000_0000):

core   0: 0x00010000 (0x00100413) li      s0, 1
core   0: 3 0x00010000 (0x00100413) x8  0x00000001
core   0: 0x00010004 (0x01f41413) slli    s0, s0, 31
core   0: 3 0x00010004 (0x01f41413) x8  0x80000000
core   0: 0x00010008 (0xf1402573) csrrs   a0, mhartid, zero
core   0: 3 0x00010008 (0xf1402573) x10 0x00000000
core   0: 0x0001000c (0x00000597) auipc   a1, 0x0
core   0: 3 0x0001000c (0x00000597) x11 0x0001000c
core   0: 0x00010010 (0x07458593) addi    a1, a1, 116
core   0: 3 0x00010010 (0x07458593) x11 0x00010080
core   0: 0x00010014 (0x00040067) jr      s0
core   0: 3 0x00010014 (0x00040067)

core   0: 0x80000000 (0x00004081) c.li    ra, 0
core   0: 3 0x80000000 (0x4081) x1  0x00000000

I also changed the execute region address base and cached region address base (in cv32_imac_sv32_config_pkg.sv file) from 0x8000_0000 to 0x0000_0000

I am using simulator as vcs-uvm, and by changing below mentioned files, the test is passing with vcs-uvm:

1. verif/env/uvme/uvme_cva6_cfg.sv (line number 158 : (!boot_addr_plusarg_valid) -> (boot_addr == 'h8000_0000);)
2. verif/tests/custom/common/test.ld
3. core/include/cv32_imac_sv32_config_pkg.sv (line 122 and 130)

In the above mentioned files I have changed the value 0x8000_0000 to 0x0000_0000.

For spike I have changed below mentioned files:

1. verif/core-v-verif/vendor/riscv/riscv-isa-sim/riscv/Proc.cc (line 200)
2. verif/core-v-verif/vendor/riscv/riscv-isa-sim/arch_test_target/spike/link.ld
3. verif/core-v-verif/vendor/riscv/riscv-isa-sim/riscv/Simulation.cc (line 50)

spike log(in case of start address 0x0000_0000): Access exception occurred while loading payload $home/cva6_git/cva6/verif/sim/out_2024-04-09/directed_c_tests/median.o: Memory address 0x130 is invalid

[zchamski]

Hi @JeanRochCoulon @SShanmukh-cell, currently Spike uses a hardwired value of reset vector base (macro DEFAULT_RSTVEC):

$ grep RSTVEC vendor/riscv/riscv-isa-sim/riscv/*.h
vendor/riscv/riscv-isa-sim/riscv/platform.h:#define DEFAULT_RSTVEC     0x00001000
[17:58 zchamski@thor2 ~/RISC-V/cva6/verif/core-v-verif] 
$ grep RSTVEC vendor/riscv/riscv-isa-sim/riscv/*.cc
vendor/riscv/riscv-isa-sim/riscv/Simulation.cc:  bus.add_device(DEFAULT_RSTVEC, boot_rom.get());
vendor/riscv/riscv-isa-sim/riscv/processor.cc:  pc = DEFAULT_RSTVEC;
vendor/riscv/riscv-isa-sim/riscv/sim.cc:  bus.add_device(DEFAULT_RSTVEC, boot_rom.get());
[17:59 zchamski@thor2 ~/RISC-V/cva6/verif/core-v-verif] 

This macro should be converted into yet another Spike parameter.

[SShanmukh-cell]

Hi @zchamski , The default RST vector is set as 0x1000, but in spike, the first PC is 0x10000. However, I have tried to change the RST vector in the vendor/riscv/riscv-isa-sim/riscv/platform.h file ( i gave value 0x80000000 to DEFAULT_RSTVEC), but still spike is starting from PC 0x10000.

Actually, I wanted the start address to be 0x00000000, but for this I am getting this error in spike : Access exception occurred while loading payload $home/cva6_git/cva6/verif/sim/out_2024-04-09/directed_c_tests/median.o: Memory address 0x130 is invalid

[zchamski]

Hi @SShanmukh-cell, you are right: our vendorized Spike version modifies the defaults using Spike params:

• /top/boot_addr overrides the default value of the PC in Processor constructor;
• /top/bootrom controls the presence of the bootROM, and /top/bootrom_base and /top/bootrom_size are used to map the bootROM memory, cf. Simulation::make_mems() and riscv/Simulation.cc:144-150.

Let me have a look at the cleanest way to alter these params to get an arbitrary start address - I'll get back to you ASAP.

[SShanmukh-cell]

Sure @zchamski, Thank you.

[SShanmukh-cell]

Hi @zchamski , Is there any update on the spike issue?

[zchamski]

Hi @SShanmukh-cell, I'm working on the fix. While testing truly arbitrary addresses I uncovered a blocking bug in Spike param management and the ETA for the full fix is now coming Friday 19 April.

[SShanmukh-cell]

Okay sure, Thanks for the update @zchamski

[zchamski]

@SShanmukh-cell, here's a quick status: I have a local fix for the blocking bug, testing is in progress (ETA unchanged). Most of the effort goes now into removing hardwired assumptions about the h'8000_0000 start address in CI infrastructure. Consequence: additional issues and PRs will need to be filed.

[SShanmukh-cell]

Sure @zchamski, thanks for looking into the issue.

[SShanmukh-cell]

Hi @zchamski Is there any update on the fix?

[zchamski]

Hi @SShanmukh-cell, the PR https://github.com/openhwgroup/core-v-verif/pull/2415 is finally out and has the necessary fixes for Spike in "solo" mode. To map a memory at specific address and force the core #0 to boot from that address, invoke Spike with the additional parameters -m<mem_base>:<mem_size> --param /top/core/0/boot_addr:uint64_t=<mem_base>, for example

-m0x40000000:0x100000 --param /top/core/0/boot_addr:uint64_t=0x40000000

For now, there's no tandem mode PR as it requires thorough modifications of RTL, CVA6 support scripts, and CI. It will arrive as part of a (future) global overhaul of platform configurability.

[SShanmukh-cell]

Hi @zchamski I have added this in verif/sim/Makefile: LD_LIBRARY_PATH="$$(realpath ../../tools/spike/lib):$$LD_LIBRARY_PATH" \ $(tool_path)/spike $(spike_stepout) $(spike_extension) --log-commits --isa=$(variant) --priv=$(priv) -l $(elf) -m0x00000000:0x40000000 --param /top/core/0/boot_addr:uint64_t=0x00000000 cp $(log).iss $(log)

But I am still getting below error: Access exception occurred while loading payload $home/cva6_git/cva6/verif/sim/out_2024-04-24/directed_c_tests/median.o: Memory address 0x130 is invalid

I have just added the specific line in verif/sim/Makefile, and I haven't changed any other file for Spike. Can you please tell me If I have made some mistake or where exactly (file ) I have add this line?

[zchamski]

The failure occurs before the beginning of payload execution - it looks like you are loading your application payload using bbl (the Berkeley Bootloader) that comes with Spike's FESVR (Front-End Server). At least that's what the error message suggests as it is probably emitted from fesvr/htif.c:101...

On the other hand, in the CVA6 verification flow we use a bare-metal setup with "spontanenous" program memory preloading, so we did not test bbl-based approach, and bbl / pk (the Proxy Kernel) are not supported because we focus primarily on core verification.

This said, the problem iseems to lie in the code of Spike's FESVR: it uses a hardwired value of entry point (DRAM_BASE, 0x80000000) in the htif class constructor at fesvr/htif.cc:47. In order to test if this is the cause, replace the DRAM_BASE with 0x0 in the constructor and ensure that the linker script for building your app places the bootloader at 0x0 rather than 0x80000000.

[JeanRochCoulon]

@SShanmukh-cell To better help you, could you submit a PR with your modifications ?

[SShanmukh-cell]

Hi @zchamski and @JeanRochCoulon, I have changed the htif.cc file (replace the DRAM_BASE with 0x0) and also changed the linker script start address, but still getting the error:

I have changed the following files and contents:

1. core\include\cv32a6_imac_sv32_config_pkg.sv : line no 122, 130
2. verif\core_v_verif\vendor\riscv\riscv_isa_sim\arch_test_target\spike-link.ld : line no  6
3. verif\core_v_verif\vendor\riscv\riscv_isa_sim\fesrv\htif.cc : line no 48
4. verif\core_v_verif\vendor\riscv\riscv_isa_sim\riscv\Params.h : as per pr 2415 
5. verif\core_v_verif\vendor\riscv\riscv_isa_sim\spike_main\spike.cc : as per pr 2415
6. verif\env\uvme\uvme_cva6_cfg.sv : line no 158
7. verif\sim\Makefile : line no 124
8. verif\tests\custom\common\test.ld 

below is the zip for the files which I have changed: spike_issue.zip

@JeanRochCoulon , I apologize as i couldn't able to create pr, hence i attached the zip

[SShanmukh-cell]

Hi @zchamski are there any updates?

[zchamski]

Hi @SShanmukh-cell, last Friday we've identified a situation where the error Access exception occurred while loading payload was raised while using the standard DRAM mapping at 0x80000000. This means that the cause of the error is not the value of DRAM_BASE but something else, possibly a missing/incorrect symbol value in the ELF file being loaded. Investigation is under way.

[SShanmukh-cell]

Thanks for the update @zchamski

[SShanmukh-cell]

Hi @zchamski, is there any update on the issue?

[dvusingh]

Hi @zchamski,

Any further updates on this issue?

Thanks

[Moschn]

Just a pointer: As far as I know, a number of registers of the debug module specified in the debug spec are often mapped to 0x0. I believe CVA6 does it this way. Therefore, if you want to map something to 0x0 you might get into issues as the debug unit will interpret your program mapped to 0x0 as inputs for its registers. The same thing applies to other address ranges, e.g., UART is at 0x1000_000, SPI at 0x2000_0000, and GPIO at 0x4000_0000. In theory, your simulator should have complained that you map both the DRAM and the debug module to the same address in the crossbar.

I have no idea about how Spike handles its memory map, though.

PS: If you have already figured all of this out, then just ignore my comment. It still might be useful for people trying to replicate this themselves.

[zchamski]

Thanks @Moschn for this input, it's good to keep it in mind. For CV32A65X the debug-at-zero is not a problem as there will be no debug block at all, but for debug-enabled configurations extra care must be taken indeed!

[SShanmukh-cell]

@Moschn and @zchamski , thanks for the inputs. I am using cv32a6_imac_sv32 configuration package and the debug module is enabled in this configuration. While running with vcs-uvm (with start address at 0x0) the test is passing, but with spike the test is failing.

[github-actions[bot]]

👋 Hi there!

This issue seems inactive. Need more help? Feel free to update us. If there are no updates within the next few days, we'll go ahead and close this issue. 😊

[zchamski]

Quick update: the necessary modifications in Spike are in the process of being validated. The corresponding PR is due out in the coming days.