Comments for "https://os.phil-opp.com/minimal-rust-kernel/"

[utterances-bot]

This is a general purpose comment thread for the “A Minimal Rust Kernel” post.

[Redrield]

I'm not really a fan of using this magic utility to turn our blob of Rust into a bootable image. The parts in raw assembly in the first edition were a bit tedious but I liked how they gave us a deeper understanding about how the process worked. I hope that this utility won't be sticking around for the post on UEFI booting...

[pixelherodev]

"They are designed to make the bootloader simple instead of the kernel. For example, the kernel needs to be linked with an adjusted default page size, because GRUB can't find the Multiboot header otherwise."

WRONG!

That is true of Multiboot2, a later standard which is less used and NOT multiboot-compatible.

However, the actual Multiboot standard does not have that problem at all.

[phil-opp]

@Redrield Fair point. On the other hand we already used grub-mkrescue as such a magic utility in the first edition, which did exactly the same thing (transforming a Rust binary into a bootable image). The only difference was that GRUB only put us into protected mode, so we had to do the last steps ourselves.

Apart from a less tedious start, the new bootloader gives us an opportunity for an even deeper understanding of the boot process than before: We're planning to write several posts on how the tool and the underlying bootloader works, in a similar fashion as the other posts (i.e. every line of code is in the post and you can follow along). Thus, we can hide the complexity at the beginning of the main post series and directly start with the more interesting part, but also provide resources to those who want to understand it all the way down.

[phil-opp]

@pixelherodev Huh? I was under the impression that Multiboot2 was the successor of Multiboot? Either way, Multiboot 1 is not a better solution for us since it does have other problems (if I remember correctly it does not support 64-bit ELF files at all).

[pixelherodev]

Fair enough. Multiboot 2 is essentially the spiritual successor to multiboot IIRC.

I also remember multiboot being easier to get started with, but that's neither here nor there :p

Honestly, while I don't think ignoring multiboot was a good idea, I can understand why you did so.

[phil-opp]

Honestly, while I don't think ignoring multiboot was a good idea, I can understand why you did so.

Yeah, there are clear tradeoffs in this case. I really like the idea behind multiboot, but the standard and the implementation not so much. I plan to add grub compatibility to the bootimage tool, so that you can boot it in a multi-OS setting too.

[ghost]

First of all, great tutorial, thanks a lot for the effort! If you're on Ubuntu, like me, and building openssl-sys when trying to cargo install bootimage fails, make sure you have libssl-dev installed!

[phil-opp]

@FuzzyHerbivore Thanks a lot! I added a hint to the post in 5f195a8.

[kylegalloway]

Disclaimer: I'm on Windows.

There is a problem with running qemu-system-x86_64 -drive format=raw,file=bootimage.bin on windows. QEMU opens and flashes but doesn't display the "Hello World!" text.

Also, converting to a VDI for VirtualBox doesn't seem to work. I get:

$ VBoxManage convertdd .\bootimage.bin bootimage.vdi --format VDI
Converting from raw image file=".\bootimage.bin" to file="bootimage.vdi"...
Creating dynamic image with size 86016 bytes (1MB)...
VBoxManage.exe: error: Cannot create the disk image "bootimage.vdi": VERR_VD_INVALID_SIZE

[AaronCoad]

@kyle I'm also using Windows with cargo 1.26.0 nightly and xargo 0.3.11. Are these the same versions for you? I'm stuck with building using xargo:

xargo build --target hyenaos results in error: multiple input filenames provided set RUST_TARGET_PATH=cd xargo build --target hyenaos results in nothing (including no build folder) as does RUST_TARGET_PATH=cd xargo build --target hyenaos.

Using cargo 1.26.0 nightly and xargo 0.3.11.

[AaronCoad]

@kylegalloway sorry

[AaronCoad]

Feeling a little silly, still stuck on the above, but realised I was setting the path incorrectly. The following steps still fail:

set RUST_TARGET_PATH=C:\HyenaOS\ or set RUST_TARGET_PATH=C:\HyenaOS xargo build --target hyenaos or xargo build --target C:\HyenaOS\hyenaos

These are always with the multiple input filenames provided error.

Apologies for the multiple posts.

[phil-opp]

@kylegalloway

There is a problem with running qemu-system-x86_64 -drive format=raw,file=bootimage.bin on windows. QEMU opens and flashes but doesn't display the "Hello World!" text.

I pushed a new version of the bootloader yesterday, seems like this is the cause of the error. I reverted the change for now, so it should work again. Thanks for reporting!

Also, converting to a VDI for VirtualBox doesn't seem to work. I get:

Hmm, I have no idea about this error. I only found https://forum.lede-project.org/t/error-convert-img-to-vdi/152, which indicates that the image needs to be padded somehow before converting.

[phil-opp]

@AaronCoad Strange, I've never seen this error. Could you try it with --verbose?

[SomeAnotherDude]

Hi, I have a little problem with bootimage. When I run it in debug mode and pass a created "bootimage.bin" file to qemu it fails with this:

    qemu-system-x86_64: Trying to execute code outside RAM or ROM at 0x000000018d47f075
    This usually means one of the following happened:

    (1) You told QEMU to execute a kernel for the wrong machine type, and it crashed on startup (eg trying to run a raspberry pi kernel on a versatilepb QEMU machine)
    (2) You didn't give QEMU a kernel or BIOS filename at all, and QEMU executed a ROM full of no-op instructions until it fell off the end
    (3) Your guest kernel has a bug and crashed by jumping off into nowhere

    This is almost always one of the first two, so check your command line and that you are using the right type of kernel for this machine.
    If you think option (3) is likely then you can try debugging your guest with the -d debug options; in particular -d guest_errors will cause the log to include a dump of the guest register state at this point.

    Execution cannot continue; stopping here.

But when I run it with --release flag everything works properly. Why could it happen?

.S my host OS is Kubuntu 17.10

[phil-opp]

@SomeAnotherDude Sounds like a problem in the bootloader. Do you have your project online somewhere so that I can try to reproduce?

[AaronCoad]

@phil-opp I gave it another whirl today and identified the issue and alerted xargo's developers to it. On my main machine my windows user account name has a space in it, so the folder path to .xargo also contains a space. Because the folder path isn't passed through to --systools with surrounding double-quotes it was picking up the folder path as being 2 separate paths. Building on secondary machine with no spaces in the username was successful.

Qemu reportedly has a bug for Windows as well in their latest version that was identified on March 22 and they're pushing a fix into rc1. http://lists.gnu.org/archive/html/qemu-devel/2018-03/msg05484.html

@kylegalloway now that I'm at that point, with VirtualBox 5.2.8 you get a more verbose error of "The given disk size 85992 is not aligned on a sector boundary (512 bytes)".

[SomeAnotherDude]

@phil-opp Uploaded to https://github.com/SomeAnotherDude/blog_os

[SomeAnotherDude]

@phil-opp It's a miracle! when I clone this repo from github and try to reproduce... It just works...

[SomeAnotherDude]

There was something wrong with ${progect_root}/target directory. Just after I deleted it and rebuilt the project the problem is gone.

[phil-opp]

@SomeAnotherDude Ah, it's probably because the bootloader is cached in the target directory. So it continued to use the old (broken) bootloader for debug and the new fixed version for release. Sorry again for the breakage.

@AaronCoad Good job in pinpointing the xargo bug! For reference, the error is tracked in https://github.com/japaric/xargo/issues/206. Unfortunately, xargo is in maintanance mode, so the author probably won't invest time in fixing this. However, they said they are still willing to merge PRs, so if you don't have time I could try to give it a shot.

@AaronCoad @kylegalloway

with VirtualBox 5.2.8 you get a more verbose error of "The given disk size 85992 is not aligned on a sector boundary (512 bytes)".

Oh, this is something we can fix in bootimage.

[phil-opp]

@AaronCoad @kylegalloway I updated bootimage in https://github.com/rust-osdev/bootimage/commit/ba6f2068dedee184665b5fce5c16116a93ba5896 to produce aligned disk images. It is on crates.io as version 0.2.0-alpha-006. You can reinstall the latest version via cargo install bootimage --force. Could you try if this fixes the VirtualBox boot?

[phil-opp]

@AaronCoad I opened https://github.com/japaric/xargo/pull/207 for the xargo bug.

[AaronCoad]

@phil-opp thankyou. I've tested the bootimage change and it is now just the invalid size error. Based on some other reported instances, it looks like it may be that VirtualBox looks for a minimum size. I reckon once it gets over that size it should convert no worries.

In the meantime, I've just setup an Ubuntu VM so that I can use qemu and look at translating and merging some of the older sections into what I've got.

[kylegalloway]

@phil-opp The new bootimage worked for me on the qemu command. Thanks for the help!

[phil-opp]

@AaronCoad @kylegalloway Cool, thanks for testing!

[mattNeumayer]

@AaronCoad @phil-opp The minimum file size for VDI seems to be 4 MiB. I've created a PR in the bootimage repo for optional padding, but you can also do this by hand with any hex editor. VMDK files seem to have no size restriction and are accepted by VirtualBox as well (but load slower imo).

I ran: VBoxManage convertfromraw bootimage.bin bootimage.vdi --format vdi --variant Standard --uuid=<some-uuid-you-generated> Without the uuid VirtualBox complains if you change the vdi file externally and then try to start the VM.

Sadly booting fails... The first stage tries to load the second stage, but after writing to 0xFFFF the next chunk is written to 0x0000 and then a interrupt is thrown. I think the A20 gate is not enabled.

[JarronAnt]

@phil-opp running the bootimage.bin on qemu doesn't seem to work qemu loads but fails saying its not a bootable device help pls

[phil-opp]

@SSPYR0 Hmm, are you passing the correct arguments to QEMU? Other than that I have no idea at the moment…

[JarronAnt]

hmm i rebuilt the binary and now it works strange

[phil-opp]

Great!

[louy2]

@mattNeumayer Thanks for your tips. I converted it successfully to a VMDK image, but booting it in VirtualBox yields the same result as yours.

EDIT: Booting in QEMU works fine. OS: macOS 10.13.4

[arjanvaneersel]

Thank you for taking the time to create such a tutorial! I'm relatively new to Rust and a complete newb to OS development and so far things were easy to follow and gave me a better understanding of how things work under the hood.

[phil-opp]

@arjanvaneersel That's really great to hear!

[Johanneslueke]

@phil-opp hello. I am at the moment going through your tutorial. It is really well made.

Right now, I have revcieved an very confusing error. If I boot the image in qemu I get notified that :

"kernel mapping failed: PageAlreadyInUse in /home/philipp/.rustup/toolchains/nightly-x86-64-unkown-linux-gnu/lib/rustlib/src/rust/src/libcore/result.rs:945"

Any Idea what this can cause? The only thing i did different was not to install bootimage 0.4.0 but instead 0.4.3. Also using cargo 1.2.8-nightly.

[Johanneslueke]

@phil-opp never mind my previous comment.... did something stupid and forgot to adjust the code from your previous post.... forgot to remove the windows startup fn

[phil-opp]

@Johanneslueke

Thanks a lot!

That's an error from the bootloader, but I'm not sure what's causing it. I tried to reproduce it, but even with the windows startup function I get a normal Hello World. But if it works now for you we don't need to investigate further.

[Bravo555]

Hello there! I happen to have the exact same issue as @Johanneslueke. Downloaded correct version of bootimage (0.4.0) an yet issue is still present. I'm using Linux and cargo 1.28.0-nightly (e2348c2db 2018-06-07).

How can I help you track the issue?

Thanks for the tutorial series! Informative and very well made.

[tiehexue]

@Bravo555 I just got same error. It turns out the main function should be named "_start". However, by the previous "freestanding" post, I followed MacOS naming which is "main".

[Bravo555]

Nevermind, I'm a massive idiot... Forgot to add #[no_mangle]... Thanks for comment, @tiehexue, without it I wouldn't have even considered function name to be a problem. Now it works fine. Thank you!

[robem]

Does anyone have a clue about this error message?

Setup (Linux):

• cargo 1.28.0-nightly (e2348c2db 2018-06-07)
• cargo-xbuild 0.4.7

$ cargo xbuild --target x86_64.json error: cannot produce bin for `os v0.1.0 (file://[..]os)` as the target `[...]x86_64.json` does not support these crate types

The crate type is bin as noted in the error message and my target is x86_64-unknown-none

UPDATE: User error

After digging into the target spec implementation I found a typo in my x86_64.json. As it turns out, executable does not equal executables. Anyhow, this way I got to spend some quality time with rustc and xargo.

[fourther]

I'm beginner of OS dev. Thank you for great tutorial! I have a question. I'd like to draw some pictures on screen. How can I change VGA text mode to standard graphic? (Some article say 0xa0000 used for standard graphic and 0xb8000... but I can't )

[robert-w-gries]

@fourther Check out OSDev Wiki's tutorial

[phil-opp]

@fourther One problem is that we only map 0xb8000 in the bootloader and not 0xa0000, so you get a page fault when trying to access that memory area. I try to fix this soon.

@robem Yeah, I made the same mistake multiple times. It's really unfortunate that there is no warning on unkown keys in the target json.

@Bravo555 @tiehexue

It turns out the main function should be named "_start". However, by the previous "freestanding" post, I followed MacOS naming which is "main".

Thanks for reporting this. I tried to clarify it in 0af14e1.

[fourther]

@robert-w-gries @phil-opp Thank you very much! I don't understand the bootloader yet, so I'll study it.

[neanias]

Hi, I'm having the same problem as @robem. I've diffed my target against the target provided in this tutorial, but they're both the same.

I'm running on macOS:

• cargo 1.28.0-nightly (e2348c2db 2018-06-07)
• cargo-xbuild v0.4.7

cargo xbuild --target x86_64-blog_os.json error: cannot produce bin for 'blog_os v0.1.0 ([...]/blog_os)' as the target '[...]/blog_os/x86_64-blog_os.json' does not support these crate types.

I've removed the panics from my Cargo.toml and renamed my main function to _start. I've also made sure that I'm specifying executables in my JSON. I can't figure it out, any ideas?

[phil-opp]

@neanias That error normally occurs when the executables key is not specified in the target json. Are you sure that you don't have a typo? Do you have your project online somewhere so that I can take a look?

[neanias]

@phil-opp, sorry for the delay. Here's the code I've got. I have an executables field, so it should be fine...

[phil-opp]

@neanias I just cloned your repository and ran cargo xbuild --target x86_64-blog_os.json. It worked without problems for me, no error at all.

[neanias]

Well, that’s not weird... I’ll give it a go on my machine again later and see what happens