search

riscvarchive / riscv-edk2-platforms

Port of EDK2 implementation of UEFI to RISC-V. See documentation at:
https://github.com/riscv/riscv-uefi-edk2-docs
Other
18 stars 7 forks source link

Create platform for QEMU's virt target #7

Closed JohnAZoidberg closed 3 years ago

JohnAZoidberg commented 3 years ago

Status

Instructions for building and runninng: https://github.com/JohnAZoidberg/riscv-edk2-docker/tree/riscv-virt

WIP branches:

Original comment

The qemu-system-riscv64 -machine virt target of QEMU doesn't look like a SiFive board. It only has virtio peripherals, therefore our current platforms don't work.

Here we don't even need our own device tree, we can just take it from MROM because QEMU puts it there: https://github.com/qemu/qemu/blob/d0ed6a69d399ae193959225cdeaa9382746c91cc/hw/riscv/boot.c#L249

JohnAZoidberg commented 3 years ago

I've modified the U540 platform for QEMU's -machine virt. It's booting to UEFI Shell, recognizes the virtio disk and starts Linux EFISTUB. But Linux get's stuck failing to get a memory page:

[    0.548367] e1000e: Intel(R) PRO/1000 Network Driver - 3.2.6-k
[    0.548876] e1000e: Copyright(c) 1999 - 2015 Intel Corporation.
[    0.549887] ehci_hcd: USB 2.0 'Enhanced' Host Controller (EHCI) Driver
[    0.550328] ehci-pci: EHCI PCI platform driver
[    0.550881] ehci-platform: EHCI generic platform driver
[    0.551453] ohci_hcd: USB 1.1 'Open' Host Controller (OHCI) Driver
[    0.551891] ohci-pci: OHCI PCI platform driver
[    0.552492] ohci-platform: OHCI generic platform driver
[    0.558739] usbcore: registered new interface driver uas
[    0.559758] usbcore: registered new interface driver usb-storage
[    0.562438] mousedev: PS/2 mouse device common for all mice
[    0.566688] goldfish_rtc 101000.rtc: registered as rtc0
[    0.567787] goldfish_rtc 101000.rtc: setting system clock to 2021-02-28T04:41:04 UTC (1614487264)
[    0.572236] syscon-poweroff poweroff: pm_power_off already claimed 0000000052f031de sbi_shutdown
[    0.573156] syscon-poweroff: probe of poweroff failed with error -16
[    0.574958] EFI Variables Facility v0.08 2004-May-17
[    0.576667] Unable to handle kernel paging request at virtual address 0000002ed0ad7be8
[    0.578052] Oops [#1]
[    0.578380] Modules linked in:
[    0.578909] CPU: 0 PID: 35 Comm: kworker/u2:1 Not tainted 5.8.0-rc1-00011-gad3b29a94a7d #754
[    0.580066] Workqueue: efi_rts_wq efi_call_rts
[    0.580517] epc: ffffffe000706b20 ra : ffffffe000706b0e sp : ffffffe03830fda0
[    0.581021]  gp : ffffffe00106a3b8 tp : ffffffe0382e1300 t0 : ffffffe037d4d000
[    0.581503]  t1 : 0000000000001873 t2 : 0000000000000000 s0 : ffffffe03830fde0
[    0.581907]  s1 : ffffffe038a91d18 a0 : ffffffe0010471f8 a1 : ffffffe037d4cc00
[    0.582337]  a2 : ffffffe038a91d20 a3 : ffffffe001018a08 a4 : 0000000000000000
[    0.582755]  a5 : 0000002ed0ad7b98 a6 : ffffffe000aa3210 a7 : 0000000000000006
[    0.583146]  s2 : ffffffe037d4cc00 s3 : ffffffe038a91d20 s4 : 0000000000000022
[    0.583622]  s5 : 0000000000000000 s6 : 0000000000000000 s7 : ffffffe0010aede0
[    0.584034]  s8 : 0000000000000001 s9 : ffffffe00021a832 s10: ffffffffffffffff
[    0.584422]  s11: ffffffe03828ecc0 t3 : 00000000fa83b2da t4 : 000000000000642f
[    0.584810]  t5 : 0000000000000000 t6 : ffffffe001078447
[    0.585107] status: 0000000000000120 badaddr: 0000002ed0ad7be8 cause: 000000000000000d
[    0.586137] ---[ end trace 59fe67af50ca423e ]---
[    0.586590] note: kworker/u2:1[35] exited with preempt_count 1

The failure looks very much like the issue here: https://lists.denx.de/pipermail/u-boot/2020-July/421194.html So perhaps we're doing something wrong with SetVirtualAddressMap. I suspect it's got something to do with PCI though, since we could boot fine on the U540 without PCI.

JohnAZoidberg commented 3 years ago

The branches are https://github.com/riscv/riscv-edk2/tree/riscv-virt-540-mod and https://github.com/riscv/riscv-edk2-platforms/tree/riscv-virt-540-mod.

OpenSBI (Silicon/RISC-V/ProcessorPkg/Library/RiscVOpensbiLib/opensbi) is at 937caee0833115f69d697ca190001ba0aa5c7368 but needs the following patch:

diff --git c/lib/utils/fdt/fdt_helper.c i/lib/utils/fdt/fdt_helper.c
index aec73a0..0ceca44 100644
--- c/lib/utils/fdt/fdt_helper.c
+++ i/lib/utils/fdt/fdt_helper.c
@@ -7,11 +7,13 @@
  */

 #include <libfdt.h>
+#include <libfdt_env.h>
 #include <sbi/riscv_asm.h>
 #include <sbi/sbi_console.h>
 #include <sbi/sbi_hartmask.h>
 #include <sbi/sbi_platform.h>
 #include <sbi/sbi_scratch.h>
+#include <sbi/sbi_string.h>
 #include <sbi_utils/fdt/fdt_helper.h>
 #include <sbi_utils/irqchip/plic.h>
 #include <sbi_utils/sys/clint.h>
@@ -123,7 +125,7 @@ int fdt_parse_hart_id(void *fdt, int cpu_offset, u32 *hartid)
        prop = fdt_getprop(fdt, cpu_offset, "device_type", &len);
        if (!prop || !len)
                return SBI_EINVAL;
-       if (strncmp (prop, "cpu", strlen ("cpu")))
+       if (sbi_strncmp (prop, "cpu", strlen ("cpu")))
                return SBI_EINVAL;

        val = fdt_getprop(fdt, cpu_offset, "reg", &len);

Building still works the same as U540, since I haven't split it out into its own platform yet:

build -a RISCV64 -p Platform/SiFive/U5SeriesPkg/FreedomU540HiFiveUnleashedBoard/U540.dsc -t GCC5 \
  && cp -v /Build/FreedomU540HiFiveUnleashed/DEBUG_GCC5/FV/U540.fd /docker/U540-virt.fd

Run it like this with your own disk image:

qemu-system-riscv64 -machine virt \
  -bios Build/FreedomU540HiFiveUnleashed/DEBUG_GCC5/FV/U540.fd \
  -m 2048 -nographic  \
  -device virtio-blk-device,drive=hd0 \
  -drive file=/home/zoid/Downloads/linux.iso,format=raw,id=hd0 \
  -device virtio-net-device,netdev=usernet \
  -netdev user,id=usernet,hostfwd=tcp::10000-:22
JohnAZoidberg commented 3 years ago

Virtio disk and virtio NIC work:

Shell> map -r
Mapping table
      FS0: Alias(s):F0:;BLK0:
          VenHw(837DCA9E-E874-4D82-B29A-23FE0E23D1E2,0080001000000000)
Shell> ifconfig -l

-----------------------------------------------------------------

name         : eth0
Media State  : Media present
policy       : static
mac addr     : 52:54:00:12:34:56

ipv4 address : 0.0.0.0

subnet mask  : 0.0.0.0

gateway: 0.0.0.0

  Routes (0 entries):

DNS server   :
JohnAZoidberg commented 3 years ago

Pre-built image based on 3a3c8579ec33335e3 at https://danielschaefer.me/static/U540-virt-3a3c8579ec33335e3.fd

kallisti5 commented 3 years ago

Hm. EDK2 definitely boots... but:

qemu-system-riscv64 -machine virt -bios ~/Downloads/U540-virt-3a3c8579ec33335e3.fd -m 2048 -device virtio-blk-device,drive=hd0 -drive file=haiku-mmc.image,format=raw,id=hd0

1) EFI/BOOT/BOOTRISCV64.EFI doesn't auto-boot 2) Bootloader doesn't start

Both of these things work fine under u-boot

logs:

UEFI Interactive Shell v2.287477C2-69C7-11D2-8E39-00A0C969723B BEDE4BA0
EDK IIlProtocolInterface: 752F3136-4E16-4FDC-A22A-E5F46812F4CA BEDE7F98
UEFI v2.70 (EDK II, 0x00010000)008-7F9B-4F30-87AC-60C9FEF5DA4E BE6B1450
Mapping table
      FS0: Alias(s):HD0b:;BLK1:
          VenHw(837DCA9E-E874-4D82-B29A-23FE0E23D1E2,0080001000000000)/HD(1,MBR,0x00000000,0x4,0x10000)
     BLK0: Alias(s):
          VenHw(837DCA9E-E874-4D82-B29A-23FE0E23D1E2,0080001000000000)
     BLK2: Alias(s):
          VenHw(837DCA9E-E874-4D82-B29A-23FE0E23D1E2,0080001000000000)/HD(2,MBR,0x00000000,0x10004,0x96000)

Press ESC in 1 seconds to skip startup.nsh or any other key to continue.
Shell> fs0:
FS0:\> cd EFI/BOOTuccess
FS0:\EFI\BOOT\> FI\BOOT' Success
FS0:\EFI\BOOT\>  Success
FS0:\EFI\BOOT\> FI' Success
FS0:\EFI\BOOT\>  Success
FS0:\EFI\BOOT\> ' Success
FS0:\EFI\BOOT\> FI\BOOT' Success
FS0:\EFI\BOOT\> 
FS0:\EFI\BOOT\> 
FS0:\EFI\BOOT\> 
FS0:\EFI\BOOT\> BOOTRISCV64.EFI'.' Success
FSOpen: Open '\EFI\BOOT\BOOTRISCV64.EFI' Success
FSOpen: Open '\EFI\BOOT\BOOTRISCV64.EFI' Success
FSOpen: Open '\EFI\BOOT\BOOTRISCV64.EFI' Success
FSOpen: Open '\EFI\BOOT\BOOTRISCV64.EFI' Success
[Security] 3rd party image[0] is deferred to load before EndOfDxe: VenHw(837DCA9E-E874-4D82-B29A-23FE0E23D1E2,0080001000000000)/HD(1,MBR,0x00000000,0x4,0x10000)/\EFI\BOOT\BOOTRISCV64.EFI.pen: Open '..' Success
InstallProtocolInterface: 5B1B31A1-9562-11D2-8E3F-00A0C969723B BEDA7940
Command Error Status: Unsupported
FSOpen: Open '.' Success
FSOpen: Open '\EFI' Success
FSOpen: Open '.' Success
FSOpen: Open '..' Success
FSOpen: Open '\EFI\BOOT' Success

Our nightly riscv64 images are available here if you want to test:

https://download.haiku-os.org/nightly-images/riscv64/

JohnAZoidberg commented 3 years ago

By the way, UEFI SCT (Self Certification Test) runs fine on the virt machine :)

vlsunil commented 3 years ago

Hi @JohnAZoidberg, Qemu has added High MMIO space (>4G) for PCIe recently. With your virt branch/prebuilt EDK2 image, it is failing to boot for this new MMIO range. Have you seen this?

ProcessPciHost: Config[0x30000000+0x10000000) Bus[0x0..0xFF] Io[0x0+0x10000)@0x3000000 Mem32[0x40000000+0x40000000)@0x0 Mem64[0x400000000+0x400000000)@0x0
CpuSetMemoryAttributes: Set memory attributes not supported yet
CpuSetMemoryAttributes: Set memory attributes not supported yet
RootBridge: PciRoot(0x0)
  Support/Attr: 70001 / 70001
    DmaAbove4G: Yes
NoExtConfSpace: No
     AllocAttr: 3 (CombineMemPMem Mem64Decode)
           Bus: 0 - FF Translation=0
            Io: 0 - FFFF Translation=0
           Mem: 40000000 - 7FFFFFFF Translation=0
    MemAbove4G: 400000000 - 7FFFFFFFF Translation=0
          PMem: FFFFFFFFFFFFFFFF - 0 Translation=0
   PMemAbove4G: FFFFFFFFFFFFFFFF - 0 Translation=0
CpuSetMemoryAttributes: Set memory attributes not supported yet

ASSERT_EFI_ERROR (Status = Not Found)
ASSERT [PciHostBridgeDxe] /edk2/MdeModulePkg/Bus/Pci/PciHostBridgeDxe/PciHostBridge.c(332): !EFI_ERROR (CheckStatus)
kallisti5 commented 3 years ago

I've noticed qemu 6.x flat out crashes when loading the latest u-boot OpenSBI bootloader. Maybe also related to the High MMIO change you mentioned?

vlsunil commented 3 years ago

@kallisti5 : I have not tried u-boot. But I expect some level of testing with u-boot done for the high mmio change. EDK2 virt support is not yet merged and hence I guess no one would have tested it.

JohnAZoidberg commented 3 years ago

No, haven't seen this. What version of QEMU?

vlsunil commented 3 years ago

It is based on v6.0.0-rc4.

vlsunil commented 3 years ago

Hi @JohnAZoidberg , I found the issue for above 4GB MMIO crash in edk2. Since it is branch, could you please review and merge below change to fix it? Basically increased the physical memory supported to 48 bits. Ideally, we should get it from the platform instead of hardcoding.

diff --git a/Platform/SiFive/U5SeriesPkg/FreedomU540HiFiveUnleashedBoard/Universal/Pei/PlatformPei/Platform.c b/Platform/SiFive/U5SeriesPkg/FreedomU540HiFiveUnleashedBoard/Universal/Pei/PlatformPei/Platform.c
index 6641e10f..3bb13548 100644
--- a/Platform/SiFive/U5SeriesPkg/FreedomU540HiFiveUnleashedBoard/Universal/Pei/PlatformPei/Platform.c
+++ b/Platform/SiFive/U5SeriesPkg/FreedomU540HiFiveUnleashedBoard/Universal/Pei/PlatformPei/Platform.c
@@ -199,7 +199,7 @@ MiscInitialization (
   // of IO space. (Side note: unlike other HOBs, the CPU HOB is needed during
   // S3 resume as well, so we build it unconditionally.)
   //
-  BuildCpuHob (32, 32);
+  BuildCpuHob (48, 32);^M
 }
JohnAZoidberg commented 3 years ago

Thanks! Would that work with QEMU 5.0 also?

vlsunil commented 3 years ago

Was this branch working with 5.0? Without my change also, it doesn't work for me. It hangs after below message.

SecCoreStartUpWithStack: Initializing OpenSBI library for booting hart 0

JohnAZoidberg commented 3 years ago

Yes, with 5.2 it was working for me. But it needs a second patch for OpenSBI that I forgot to mention here: https://github.com/riscv/riscv-edk2-platforms/issues/14#issuecomment-798828632

JohnAZoidberg commented 3 years ago

I created a fork of OpenSBI to put our patches and updated the submodule to point to there. No need for manually patching OpenSBI anymore.

Just do

git pull
git submodule sync
git submodule update --init
JohnAZoidberg commented 3 years ago

Thanks! Would that work with QEMU 5.0 also?

Ok, I can confirm that your proposed change works with 5.2 and 6.0. Thanks! I pushed it to the branch.

JohnAZoidberg commented 3 years ago

This branch doesn't work with older QEMU. It hangs directly after:

SecCoreStartUpWithStack: Initializing OpenSBI library for booting hart 0

so I don't think this is due to the change contributed by @vlsunil.

Only Ubuntu 21.04 has 5.2. Older version have only 5.0 or older.

JohnAZoidberg commented 3 years ago

I consider this done. It's not upstreamed but it's working well on riscv-virt-gh-actions branch.

Just build Platform/Qemu/RiscvVirt/RiscvVirt.dsc or download the RISCVVIRT.fd from the latest run of GitHub actoins.

Linux boots fine from ESP on virtio :)