cnrv / fpga-rocket-chip

Wrapper for Rocket-Chip on FPGAs
Other
124 stars 26 forks source link

How to enable multi core? #17

Closed K16DIABLO closed 3 years ago

K16DIABLO commented 3 years ago

I want to make the dual core rocket, but it fails during 1st BL stage. Even printf fails. When single core, FSBL is printed, but dual core, FFSSBBLL is printed. It cannot open file.

Also, is there any way to enable 64bit system? Firmware cannot copy the bootloader to DRAM when virtual address over 32bit.

wsong83 commented 3 years ago

By your description, I think you have already figured out how to add the second core in hardware, nice job!

When single core, FSBL is printed, but dual core, FFSSBBLL is printed. It cannot open file.

When you have multiple core initiated, one of them must be initiate as the main core (running the FSBL) while other cores should wait until some jobs are allocated to them (normally after the Linux kernel is initiated). A simple way to do this is to set different start addresses to different cores; therefore, each core runs its own FSBL. Here, only the main core runs a real FSBL while other cores simply waiting for interruptions (WFI I suppose).

Also, is there any way to enable 64bit system? Firmware cannot copy the bootloader to DRAM when virtual address over 32bit.

By default, it is already a 64-bit system (allow 64-bit address). Your question is a little bit mixed up. For the hardware, the DDR does not understand virtual address. All addresses used fro accessing caches and memory are physical addresses and they are normally much narrower than 64-bit. The 64-bit virtual addresses (normally 39-bit or 48-bit) need to be translated (by MMU) to physical addresses when accessing the DDR. So here the question is:

K16DIABLO commented 3 years ago

Do I have to put device tree in FSBL? How the firmware knows which core is the main core?

wsong83 commented 3 years ago

Although not totally sure, I believe normally it is hardwired unless you are worry about that the statically chosen core might fail due to imbalanced load.

K16DIABLO commented 3 years ago

image I think I should post the screenshot. This is what happens when I use dual core. I'm not sure what is happening, but it seems both cores try to run same FSBL and fail. How can I set different address to each core? Device tree is not included in FSBL..

K16DIABLO commented 3 years ago

Have you correctly set up the initial page table when trying to copy stuff to DDR using virtual addresses?

I think I have. I modified the address range in subsystem/Config.scala in rocket-core directory image

Also modified the DDR size in sdload.c in firmware directory image

However, when I use this rocket-core, firmware stalls in line 71 in the figure below (firmware/driver/spi.c) image It fails to copy data to destination.

This does not happen when I use configuration: image I left everything without rocket-core unchanged.

wsong83 commented 3 years ago

It seems like you have misunderstood what I said about the core initialization. This is not about hardware but software!

As I understand only the BBL, I will use it as an example. For the part that "a single core is initiated", see code related to: https://github.com/riscv/riscv-pk/blob/62bb5daea5ef014616b00a63c106afdd07e68ffd/machine/mentry.S#L288 https://github.com/riscv/riscv-pk/blob/62bb5daea5ef014616b00a63c106afdd07e68ffd/machine/minit.c#L172

For setting up the page table, see: https://github.com/riscv/riscv-pk/blob/0011fdf21192bd97b96c078c56cea07059c64f04/pk/mmap.c#L393

NeutrinoLiu commented 3 years ago

months ago i succeeded to boot linux on a 4-core boom based on this repo. As far as i remember, I only changed the description of dts file (https://github.com/cnrv/riscv-pk/blob/97d7482bde61d39be880c45a717fdc414b057184/build/temporary_dtb.dts#L14), and everything else works fine for me. btw, if you make any modification to the address, plz review the following files: riscv-pk/build/temporary_dtb.dts, fpga-rocket-chip/firmware/*

wsong83 commented 3 years ago

months ago i succeeded to boot linux on a 4-core boom based on this repo. As far as i remember, I only changed the description of dts file (https://github.com/cnrv/riscv-pk/blob/97d7482bde61d39be880c45a717fdc414b057184/build/temporary_dtb.dts#L14), and everything else works fine for me. btw, if you make any modification to the address, plz review the following files: riscv-pk/build/temporary_dtb.dts, fpga-rocket-chip/firmware/*

Great! Nice to know.