QEMU Compatibility Issues

[torrancew]

Note: I have not yet tested on a physical Raspberry Pi. All testing has been under qemu, invoked as follows: qemu-system-arm -machine raspi -cpu arm1176 -nographic -sd /path/to/sd.img -kernel kernel-qemu.img

I am working with the following stack:

ArchLinux
arm-none-eabi-gcc 5.2.0-1
arm-none-eabi-binutils 2.25.1-2
arm-none-eabi-gdb 7.10-2
qemu: Torlus's rpi branch

Unfortunately, I've had a number of problems trying to move a home-grown kernel based on the OSDev Wiki to this bootloader.

After some issues, I have applied the following patches, which seem to have improved things (but this could be as much illusion as reality):

diff --git a/Makefile.rpi-boot.in b/Makefile.rpi-boot.in
index 1ba4de7..39b5084 100755
--- a/Makefile.rpi-boot.in
+++ b/Makefile.rpi-boot.in
@@ -13,7 +13,7 @@ DISASM_DUMP = kernel.list

 all: kernel.img

-CFLAGS += -pedantic -pedantic-errors -nostdlib -nostartfiles -ffreestanding -Wall -Wextra -Werror -Wshadow
+CFLAGS += -pedantic -pedantic-errors -nostdlib -ffreestanding -Wall -Wextra -Wshadow
 CFLAGS += -std=gnu99
 CFLAGS += -I.
 CFLAGS += -DBUILDING_RPIBOOT
diff --git a/config.h b/config.h
index be54e30..4fb11e1 100755
--- a/config.h
+++ b/config.h
@@ -10,7 +10,7 @@
    CFLAGS=-DEMMC_DEBUG make 

    Also affects compiler optimization flags */
-#undef DEBUG
+#define DEBUG

 /* Enable the framebuffer as an output device */
 #define ENABLE_FRAMEBUFFER
diff --git a/emmc.c b/emmc.c
index 29552df..cde40fa 100644
--- a/emmc.c
+++ b/emmc.c
@@ -79,7 +79,7 @@
 // The particular SDHCI implementation
 #define SDHCI_IMPLEMENTATION_GENERIC        0
 #define SDHCI_IMPLEMENTATION_BCM_2708       1
-#define SDHCI_IMPLEMENTATION                SDHCI_IMPLEMENTATION_BCM_2708
+#define SDHCI_IMPLEMENTATION                SDHCI_IMPLEMENTATION_GENERIC

 static char driver_name[] = "emmc";
 static char device_name[] = "emmc0";   // We use a single device name as there is only
diff --git a/linker-qemu.ld b/linker-qemu.ld
index 41debb8..423eac7 100644
--- a/linker-qemu.ld
+++ b/linker-qemu.ld
@@ -37,6 +37,14 @@ SECTIONS
    }
    . = ALIGN(4096);
    _bss_end = .;
+
+   __exidx_start = .;
+   .ARM.exidx :
+   {
+       *(.ARM.exidx*)
+   }
+   __exidx_end = .;
+
    _end = .;
 }

diff --git a/malloc.c b/malloc.c
index 966b360..1a82cd3 100644
--- a/malloc.c
+++ b/malloc.c
@@ -2795,7 +2795,7 @@ static void reset_on_error(mstate m);

 /* -------------------------- Debugging setup ---------------------------- */

-#if ! DEBUG
+#ifndef DEBUG

 #define check_free_chunk(M,P)
 #define check_inuse_chunk(M,P)
@@ -3216,7 +3216,7 @@ static int change_mparam(int param_number, int value) {
   }
 }

-#if DEBUG
+#ifdef DEBUG
 /* ------------------------- Debugging Support --------------------------- */

 /* Check properties of any chunk, whether free, inuse, mmapped etc  */
@@ -5071,7 +5071,7 @@ static void** ialloc(mstate m,
     }
   }

-#if DEBUG
+#ifdef DEBUG
   if (marray != chunks) {
     /* final element must have exactly exhausted chunk */
     if (element_size != 0) {

Respectively:

• CFLAGS were adjusted to resolve gcc errors related to system.h
• DEBUG was defined to aid in the investigation. Several checks for the same were tweaked to fix apparent bugs that expected DEBUG to be set to an actual value.
• Once DEBUG was defined, kernel-qemu.elf was unbuildable, until patching linker-qemu.ld
• After all of the above, I determined that I was having issues with the EMMC drivers - specifically, if((mailbuffer[6] & 0x3) != 1), located in bcm_2708_power_on of emmc.c. Realizing I was out of my depth, I swapped to the apparent generic implementation, which allowed the config to be found and printed by the loader.

All of the above has worked only on EXT2 filesystems - FAT filesystems have simply not worked for me, using the provided mksdimg.sh script, or otherwise. Even with EXT2 filesystems, however, the loader simply hangs after printing the discovered config file. After poking around with GDB, I have determined that fread is not functioning properly (and possibly malloc, but have been unable to confirm with certainty), but obviously am not sure if this is due to my muckery or not. Any assistance you could provide would be appreciated.

[jncronin]

Thanks for the report. Unfortunately I probably wont be able to properly investigate this for a while due to time pressures. A few things jump out at me though:

1) You mention the build pulls in 'system.h' which causes problems with -Werror. Where is this file? Is it part of gcc's headers pulled in by something else? It really shouldn't be included in the build if -ffreestanding is used. 2) The malloc implementation is dlmalloc, a widely used public domain malloc. It shouldn't have any obvious bugs (but I can't say for sure). 3) The requirement for the linker.ld changes probably reflects changes in a recent gcc (I last built rpi-boot with something around version 4.5) and needs adding to the repository 4) The bcm_2708_power_on functions are used to provide a clean reset on real hardware. I'd guess the sdhci controller in the Torlus qemu fork doesn't support them yet. Its an easy fix to use the default implementation on qemu. 5) When you say 'fread is not functioning properly', do you mean the actual fread() or fat_fread()? In particular, if you build with either DEBUG2 or FAT_DEBUG defined, you will get more debugging info on the fat system.

[torrancew]

@jncronin Thank you for the response - understood re:timing and commitments.

To address your feedback:

1. For some reason, I cannot reproduce this on master, and am dropping this change (initial builds were via clang, perhaps I confused myself during that time)
2. ACK. I was uncertain if a problem existed or not, and am leaning towards not
3. ACK. Will file a PR - I find it curious though that linker.ld already had this, while linker-qemu.ld did not.
4. More or less what I suspected. Can I take this as confirmation that the generic SD implementation is expected to work?
5. Allow me to elaborate. I am consistently having trouble with the storage layer. FAT does not seem to function at all. No config is found. Under EXT2, my config is found, but the loader hangs after reporting it. Further inspection with gdb shows that the fread calls are appearing to work (ie they return a size that reflects the expected data), but the buffers are not being populated. The end result is that the config file contents buffer passed to cfg_parse is an empty string. The same is true for first_8k in multiboot.c. I suspect this is happening in ext2_fread or lower, but again, am a bit out of my depth there. If you have any suggestions that could help me chase this down, I'd greatly appreciate it.

Thank you for your work on this project!

[torrancew]

Update: I have gotten a successful boot with the following

• As above, swap the default EMMC implementation to the generic one
• For the boot SD, use a FAT16 partition

As such, I believe it is safe to say the following:

• FAT32 is not currently supported (or if expected to be, may be buggy)
• EXT2 support is buggy, at least in the face of using the generic EMMC implementation

I will follow up with a PR or two to merge in the more valuable changes (fixing the debug checks to use ifdef/ifndef, fixing linker-qemu.ld, and possibly using a flag in config.h to allow the QEMU builds to select the generic EMMC implementation more easily. I will also try to chase down what is going wrong with EXT2 here, but that may be a bit beyond my abilities.

Thank you!

[jncronin]

Sorry for the late reply. Have you got any further? If not, could you provide a copy of a filesystem image that is not working for you and I'll have a look at it.

[jncronin]

I've pushed a few more fixes for some of the issues you've identified. Could you test again and let me now.

[AmbiBox]

I am consistently having trouble with the storage layer FAT does not seem to function at all. No config is found.<<

My fix (27.11.2015): emmc.c:

define SD_CLOCK_LOW 12500000

sd_switch_clock_rate(base_clock, SD_CLOCK_LOW); //SD_CLOCK_NORMAL

main.c: static char *boot_cfg_names[] = { "/rpi_boot/rpi-boot.cfg", "rpi_boot/rpi-boot.cfg", "rpi-boot.cfg", 0 };

P.S. first partition have label "boot" and first partition (FAT16) contain folder "boot". RPI BOOT file system find both "boot" (label and folder) and attempting to read data from a label. Label "boot" not folder !!! Therefore it is necessary change the disk label to the other or change the folder name.

Even with EXT2 filesystems, however, the loader simply hangs after printing the discovered config file. <<

if you use test.elf from the repository, it displayed on the screen "Welcome to the test kernel", if to compile the test.elf does not work !!!

[torrancew]

@jncronin With the changes you pushed, I no longer need the linker fixes for QEMU.

However, since I opened this request, upstream QEMU has pulled in support for the Pi2 via the "raspi2" machine type (unsure which version this first appeared in, but it is available in the 2.6.0 that Arch packages). rpi-boot does not work with this QEMU machine, and (for me) seems to hang around the fb_init phase -- would you have the time/motivation to take a peek at that, perchance, or point me down some likely avenues of investigation?

[jncronin]

Hi, I had a quick look. By 'hang around the fb_init phase' do you mean it outputs stuff to the screen, then stops? I think the base address of the SDHCI has changed in rpi2. The official way to do it is to parse the device tree handed to rpi-boot by the firmware as apparently it can change per device. Unfortunately I don't have a rpi2 to test with. Having had a quick look through the qemu emulation this morning, however, I think the sdhci in qemu is hardwired to at address 0x3f300000, so changing line 129 in emmc.c to #define EMMC_BASE 0x3f300000 may help.

[torrancew]

Great, I'll give that a go!

[torrancew]

@jncronin Thank you for the tip. That was very helpful!

With the adjusted memory offset (needed in several locations), I was able to get to the point of the usleep call in emmc.c. It turns out that the current raspi2 emulation in QEMU does not emulate the BCM2835 System Timer used by timer.c, so I've got a local branch where I'm working on that support. I've got rpi-boot working (somewhat) with the new machine type, but still have a few kinks in the timer code to work out. Once I do, I'll submit that upstream to QEMU, and work on a patch for Pi2 support here, as well. I'll likely target the default QEMU addresses to start, and then work on DTB support from there.

[jncronin]

I've added some basic support for device tree (just used for getting memory size for now, and using different addresses for emmc/mbox/uart if a model 2 is detected). Note that you actually have to pass a .dtb file to qemu (via -dtb) to get this to work. I've also added a workaround in the timer code so that if a timer is not detected then it doesn't wait (on the presumption that delays are required for an emulator anyway).