search

chipsalliance / dromajo

RISC-V RV64GC emulator designed for RTL co-simulation
Apache License 2.0
210 stars 63 forks source link

Maximum core count greater than 8? #49

Open muwyse opened 2 years ago

muwyse commented 2 years ago

Background

I am working to emulate/simulate RISC-V systems (using multiple tools) with more than 8 cores and booting SMP Linux with the OpenSBI bootloader. I currently have a setup that works with both Dromajo and QEMU (v6.1.0) to boot OpenSBI v0.9 with Linux Kernel v5.9 and a simple Buildroot/Busybox filesystem. This system successfully boots in Dromajo with up to 8 cores, but fails at higher core counts due to the hard-coded core count limit of 8 in Dromajo (https://github.com/chipsalliance/dromajo/blob/3675ba09996dbe24286af45a969443a0ee3cfe6e/include/riscv_machine.h#L51).

I attempted to modify the source code and increase this limit, say to 32, but running with anything more than 8 cores results in either an error (e.g, with 9 cores: riscv_cpu_read_memory: invalid physical address 0x0000000000011000) or a segmentation fault of Dromajo itself. One potential conclusion is that the Dromajo generated device tree is invalid for core counts > 8 and it isn't allocating a large enough memory range for some device.

Questions

What is the motivation for limiting Dromajo to simulating 8 cores maximum and what are the constraints or requirements for increasing the maximum core count?

Are there any short-term workarounds for running with > 8 cores (e.g., if this is indeed a device tree issue and it is possible to provide a custom device tree)?

Branch/Commit

master at 3675ba09996dbe24286af45a969443a0ee3cfe6e

Misc

Note that unmodified QEMU v6.1.0 has a similar hard-coded core limit for the riscv64-virt machine, but increasing this limit and rebuilding QEMU seems to cause no problems.

et-tommythorn commented 2 years ago

There is not particular motivation, it was just a limit set somewhere and it would be interesting to raise it in a way that doesn't penalize lower, or even single, core simulations.

Regarding the device tree, it doesn't seem that it should be very hard to expand it, but there's a bigger question of whether Dromajo really should be producing this rather than consuming it (or both potentially).

TL;DR: I'd be very interested in pull request.