search

tosc-rs / mnemos

An Operating System for Building Small Computers
https://mnemos.dev
Apache License 2.0
253 stars 18 forks source link

build(d1): automate building D1 oreboot SD card images #273

Open hawkw opened 1 year ago

hawkw commented 1 year ago

there are some instructions in https://github.com/tosc-rs/mnemos/blob/main/platforms/allwinner-d1/README.md explaining how to make a bootable SD card for the D1 using MnemOS + Oreboot. currently, this is accomplished by using dd to write the oreboot and mnemOS binaries directly to the SD device.

it would be cool if we could add some automation for producing a bootable ISO image or similar that can then be written to a SD card, similar to how our build process can produce x86_64 bootable images. ideally, this would:

hawkw commented 1 year ago

started on this; i was hoping we could do something similar to the mnemos-x86_64-bootloader crate and have a bindep on the actual kernel binary, but this unfortunately runs into some cargo bugs, so we might need to go with an xtask approach instead.

hawkw commented 1 year ago

also, we probably need to download prebuilt oreboot artifacts from GitHub Releases or something — we can't depend on oreboot as a bin dep, because it needs post-build processing, which I believe oreboot does with an xtask.

hawkw commented 1 year ago

cc @jspngh, you might be interested in this as well?

jspngh commented 1 year ago

Yes, indeed! I think the easiest way of getting the bootloader is downloading a prebuilt binary from somewhere, for the reasons you mentioned. I should be able to add releases to my oreboot fork to make it available there. Or maybe it's possible to create a pipeline that builds on every push and to always take the latest artifact?

I'm not too far away from reading data from SD cards in mnemos itself, so it would be nice to have a mnemos-d1-bootlader. But the first stage bootloader needs to fit in 32K, which may not be feasible looking at the current size of the D1 binaries.

Anyway, having a self-contained iso file that can be used like usual (just dd to SD card without extra argument) would be awesome. What I would put in there is:

Then when a user plugs in the SD card into the computer, they can work with the filesystem (e.g., add forth scripts?).

hawkw commented 1 year ago

@jspngh oh, that's fantastic, i didn't realize you were that far along on reading from the SD card!

the eventual goal is definitely to be able to build an image with a filesystem and put some user mode programs on there as well (maybe a forth program launcher/task browser, and/or an init system type thing). so, a way to provide other files you want on the fs is probably the next step once we have a nice way to build a bootable ISO!

hawkw commented 1 year ago

I'm not too far away from reading data from SD cards in mnemos itself, so it would be nice to have a mnemos-d1-bootlader. But the first stage bootloader needs to fit in 32K, which may not be feasible looking at the current size of the D1 binaries.

i think we could probably produce a significantly smaller mnemos binary, with a little work. the D1 kernel binaries aren't built with opt-level='s', for one thing, and they include a bunch of code we wouldn't need in a first stage, like the TWI, i2c_puppet, and keyboard drivers. we could probably statically disable the more verbose tracing diagnostics in a first stage build as well.

if we can't jam a size-optimized kernel with most of that stuff disabled into 32k, though, we could also consider writing a separate bootloader that doesn't use the actual mnemos kernel. the async runtime probably results in generating a lot of code. since a stage 0 wouldn't be running arbitrary user tasks (it's just doing DRAM initialization, loading the kernel from SD, and maybe printing stuff to the UART for debugging), we could maybe use some of the existing platform HAL stuff we've written but leave behind the async executor, timer wheel, and other big stuff like that.

but, for now, oreboot works fine too, so my first pass at a boot image builder will just use your oreboot fork.

jspngh commented 1 year ago

I've added a release, we should be able to start using this URL https://github.com/jspngh/oreboot/releases/download/v0.1.0-mnemos/oreboot-nezha-bt0.bin

If you have any suggestions for improvement, I'm happy to hear it.

if we can't jam a size-optimized kernel with most of that stuff disabled into 32k, though, we could also consider writing a separate bootloader that doesn't use the actual mnemos kernel. the async runtime probably results in generating a lot of code. since a stage 0 wouldn't be running arbitrary user tasks (it's just doing DRAM initialization, loading the kernel from SD, and maybe printing stuff to the UART for debugging), we could maybe use some of the existing platform HAL stuff we've written but leave behind the async executor, timer wheel, and other big stuff like that.

That would be neat, especially if we could use the existing drivers. I don't know how it would work without an async executor though, since the drivers also contain async functions?

but, for now, oreboot works fine too, so my first pass at a boot image builder will just use your oreboot fork.

Yep, that sounds like the easiest way forward.

jspngh commented 1 year ago

I've added a release, we should be able to start using this URL https://github.com/jspngh/oreboot/releases/download/v0.1.0-mnemos/oreboot-nezha-bt0.bin

Now we can also get rid of the oreboot-bt0.tar.gz in the repo root, I think you may have accidentally committed it with some other files :grin:

hawkw commented 1 year ago

I've added a release, we should be able to start using this URL https://github.com/jspngh/oreboot/releases/download/v0.1.0-mnemos/oreboot-nezha-bt0.bin

lovely, thank you! i haven't had much time to spend on the actual build script yet, but this will be very helpful!