Comments for "https://os.phil-opp.com/freestanding-rust-binary/"

[utterances-bot]

This is a general purpose comment thread for the “A Freestanding Rust Binary” post.

[sunjay]

There is a broken link which isn't actually formatted as a link: [name mangling][mangling]

[phil-opp]

Thanks! Fixed in 8bd66bf.

[memoryruins]

Both the first and second editions of these series are invaluable. Thank you for every bit of clearly written information you’ve shared ✨

[phil-opp]

@memoryruins Thanks so much!

[CornedBee]

The Windows subsystem information link is outdated. The page behind it is pretty much unreadable and suggests to link to here instead:

https://docs.microsoft.com/en-us/cpp/build/reference/entry-entry-point-symbol

[phil-opp]

@CornedBee Thanks for the hint! Fixed in 6599a69.

[AleVul]

In the windows part of the post, the CONSOLE subsystem entry points are used as an example, then immediately the Windows subsystems entry points are referenced, i got a feeling that some explication is missing there.

[phil-opp]

@AleVul Thanks for reporting! We used to define the Windows subsystem entry points and then switched to console subsystem entry points. Seems like we missed some references. Fixed in 5f12cd8.

[sepiropht]

Is the final code sample is supposed to build wihtout errors ? Compilator ask me to import PanicInfo

[phil-opp]

@sepiropht Thanks for reporting. Seems like we forgot to add the imports in the latest update. I opened #440 to fix this.

[dodikk]

@phil-opp , thanks for a great series. Really well written and well explained (so much beginner-friendly).

What do you think about adding the platform annotations for the trampoline functions?

#[cfg(target_os = "windows")]
#[no_mangle]
pub extern "C" fn mainCRTStartup() -> ! {
    main();
}

[phil-opp]

@dodikk

thanks for a great series. Really well written and well explained (so much beginner-friendly).

Thanks a lot!

What do you think about adding the platform annotations for the trampoline functions?

These functions are just used to get a compiling no_std executable on existing operating systems in the first post. It wouldn't make sense to keep them for the subsequent posts because our OS kernel won't ever run on top of Windows or macOS. Only adding the conditional compilation for the first post would be possible, but I don't think that it's worth it because it makes things more complicated and the functions are removed in the second post anyway.

[AregevDev]

But you have two main() functions, one for Windows and one for MacOS. It errors on it, am I missing something?

[phil-opp]

@AregevDev You should only use one of the three variants, depending on your host OS. So if you're on Windows, only add the Windows-style entry point and not the Linux and macOS entry points.

[andre-richter]

@AregevDev An alternative approach to having os-specific entrypoints would be the introduction of a custom linker script. There, you can define the entry-point function via ENTRY().

I think the first edition of the blog talked about it.

One way or the other, you have a little piece of inconvenience. When writing a kernel, though, in the long run you will probably end up with a custom linker script at some point anyways.

[jaypatelani]

panic_implementation : this attribute was renamed to panic_handler. See https://github.com/rust-lang/rust/issues/44489#issuecomment-415140224

[jessejhein]

In addition to the panic_implementation being renamed, using the new name means you don't need the #![feature(panic_implementation)] anymore (or I believe the nightly?).

[phil-opp]

@jaypatelani @jessejhein Thanks for reporting. I updated the blog in https://github.com/phil-opp/blog_os/pull/468.

[cymno]

As of rustc 1.29.2 stable it is still necessary to use the nightly toolchain on Linux (and probably Mac), as the '-Z' option works only in nightly.

[fcogama]

Hi, playing around I'm getting the following linker error with Rust nightly build 1.31.0.

rustc 1.31.0-nightly (3e6f30ec3 2018-10-26)

error: linking with `cc` failed: exit code: 1                                                                                                                                                                      
  |                                                                                                                                                                                                                
  = note: "cc" "-Wl,--as-needed" "-Wl,-z,noexecstack" "-m64" "-nostartfiles" "-L" "/home/devop/.rustup/toolchains/nightly-x86_64-unknown-linux-gnu/lib/rustlib/x86_64-unknown-linux-gnu/lib" "/home/devop/workspace/tactical_os/target/debug/deps/tactical_os-091a9ff38ca58526.1lwzcx53g884pkrj.rcgu.o" "/home/devop/workspace/tactical_os/target/debug/deps/tactical_os-091a9ff38ca58526.1mxek59mmosxp1gr.rcgu.o" "/home/devop/workspace/tactical_os/target/debug/deps/tactical_os-091a9ff38ca58526.3vwhvtx17707okjv.rcgu.o" "/home/devop/workspace/tactical_os/target/debug/deps/tactical_os-091a9ff38ca58526.4k2fyypy62ab6aaa.rcgu.o" "/home/devop/workspace/tactical_os/target/debug/deps/tactical_os-091a9ff38ca58526.pmfm3pkes9s56eu.rcgu.o" "/home/devop/workspace/tactical_os/target/debug/deps/tactical_os-091a9ff38ca58526.u78q3renarfctfb.rcgu.o" "-o" "/home/devop/workspace/tactical_os/target/debug/deps/tactical_os-091a9ff38ca58526" "/home/devop/workspace/tactical_os/target/debug/deps/tactical_os-091a9ff38ca58526.36u6sdy3tifj0l3t.rcgu.o" "-Wl,--gc-sections" "-pie" "-Wl,-zrelro" "-Wl,-znow" "-nodefaultlibs" "-L" "/home/devop/workspace/tactical_os/target/debug/deps" "-L" "/home/devop/.rustup/toolchains/nightly-x86_64-unknown-linux-gnu/lib/rustlib/x86_64-unknown-linux-gnu/lib" "-Wl,--start-group" "-Wl,-Bstatic" "/home/devop/.rustup/toolchains/nightly-x86_64-unknown-linux-gnu/lib/rustlib/x86_64-unknown-linux-gnu/lib/libstd-749cdbe450dd7140.rlib" "/home/devop/.rustup/toolchains/nightly-x86_64-unknown-linux-gnu/lib/rustlib/x86_64-unknown-linux-gnu/lib/libpanic_abort-b02d3fe29f436127.rlib" "/home/devop/.rustup/toolchains/nightly-x86_64-unknown-linux-gnu/lib/rustlib/x86_64-unknown-linux-gnu/lib/liballoc_jemalloc-cf548d84ef70130f.rlib" "/home/devop/.rustup/toolchains/nightly-x86_64-unknown-linux-gnu/lib/rustlib/x86_64-unknown-linux-gnu/lib/libunwind-1d3e911ab051fda4.rlib" "/home/devop/.rustup/toolchains/nightly-x86_64-unknown-linux-gnu/lib/rustlib/x86_64-unknown-linux-gnu/lib/liballoc_system-b9cd5a1dfd88584e.rlib" "/home/devop/.rustup/toolchains/nightly-x86_64-unknown-linux-gnu/lib/rustlib/x86_64-unknown-linux-gnu/lib/liblibc-8f294fb62aa82289.rlib" "/home/devop/.rustup/toolchains/nightly-x86_64-unknown-linux-gnu/lib/rustlib/x86_64-unknown-linux-gnu/lib/liballoc-b6501604988320ff.rlib" "/home/devop/.rustup/toolchains/nightly-x86_64-unknown-linux-gnu/lib/rustlib/x86_64-unknown-linux-gnu/lib/libcore-56f5587bc5216ac9.rlib" "-Wl,--end-group" "/home/devop/.rustup/toolchains/nightly-x86_64-unknown-linux-gnu/lib/rustlib/x86_64-unknown-linux-gnu/lib/libcompiler_builtins-e578c298dc2b65d9.rlib" "-Wl,-Bdynamic" "-ldl" "-lrt" "-lpthread" "-lpthread" "-lgcc_s" "-lc" "-lm" "-lrt" "-lpthread" "-lutil" "-lutil"
  = note: /usr/bin/ld: warning: cannot find entry symbol _start; defaulting to 0000000000003370                                                                                                                    
          /usr/lib/x86_64-linux-gnu/libpthread_nonshared.a(pthread_atfork.oS): In function `__pthread_atfork':                                                                                                     
          /build/glibc-OTsEL5/glibc-2.27/nptl/pthread_atfork.c:51: undefined reference to `__dso_handle'                                                                                                           
          /usr/lib/x86_64-linux-gnu/libc_nonshared.a(atexit.oS): In function `atexit':                                                                                                                             
          (.text+0x3): undefined reference to `__dso_handle'                                                                                                                                                       
          /usr/bin/ld: /home/devop/workspace/tactical_os/target/debug/deps/tactical_os-091a9ff38ca58526: hidden symbol `__dso_handle' isn't defined                                                                
          /usr/bin/ld: final link failed: Bad value                                                                                                                                                                
          collect2: error: ld returned 1 exit status                                                                                                                                                               

error: aborting due to previous error                                                                                                                                                                              

error: Could not compile `tactical_os`.                                                                                                                                                                            

To learn more, run the command again with --verbose.

[phil-opp]

@fcogama I never seen this one before. Are you using Linux? If yes, are you using a #[no_mangle] _start function and compiling with -Z pre-link-arg=-nostartfiles?

[phil-opp]

As of rustc 1.29.2 stable it is still necessary to use the nightly toolchain on Linux (and probably Mac), as the '-Z' option works only in nightly.

@cymno Sorry for the late reply! You're absolutely right. I opened https://github.com/phil-opp/blog_os/pull/493 to fix this.

[lilyball]

With current rustc nightly, the errors printed with no panic handler defined no longer references panic_impl but instead says:

> cargo build
error: `#[panic_handler]` function required, but not found
error: language item required, but not found: `eh_personality`

[phil-opp]

@kballard Thanks! I updated the code and reworded the section in 298e714.

[guncha]

This was super useful to get me started. Once I did, I ran into some seemingly arbitrary segfaults. Turns out the libc initialization does something useful after all! It aligns the stack to 16 bytes which is necessary for SSE instructions. So if you find that you're reliably getting SIGSEGV when executing xmmN instructions which can happen when destructuring tuples or passing large stack allocated arguments to a function, this is why.

#[no_mangle]
fn main() {
  // the actual main
}

#[no_mangle]
pub unsafe extern "C" fn _start() {
    // Align the stack to the 16 byte boundary as the stdlib C preamble would
    asm!(
        "
        and $$0xfffffffffffffff0, %rsp;
        call main;
    "
    );
}

[phil-opp]

@guncha Yeah, the x86_64 architecture requires that the stack is aligned on each function entry. The bootloader that we add in the next post should do this for our kernel.

[mfournial]

Adding the compiler flags in a .cargo/config in the root of the project like so:

[build]
rustflags = ["-C", "link-arg=-nostartfiles"]

keeps the build commands consistent accross OSes (cargo build)

[reesmanp]

Just because I might develop on multiple devices which are all different OS's it makes sense to have conditional compiling (I might be developing on my desktop windows machine one day and then my mac laptop the next).

main.rs

#![no_std]
#![no_main]

use core::panic::PanicInfo;

#[panic_handler]
fn panic(_info: &PanicInfo) -> ! {
    loop {}
}

/// For development on Linux
#[cfg(target_os = "linux")]
#[no_mangle]
pub extern "C" fn _start() -> ! {
    loop {}
}

/// For development on Windows
#[cfg(target_os = "windows")]
#[no_mangle]
pub extern "C" fn mainCRTStartup() -> ! {
    main();
}

/// For development on Windows or MacOS
#[cfg(any(target_os = "windows", target_os = "macos"))]
#[no_mangle]
pub extern "C" fn main() -> ! {
    loop {}
}

.cargo/config

[target.'cfg(target_os = "linux")']
rustflags = ["-C", "link-arg=-nostartfiles"]

[target.'cfg(target_os = "macos")']
rustflags = ["-C", "link-arg=-lSystem"]

[zhiyuan-liao]

@andre-richter Hi, I'm using osx. But I found I could pass the -static flags to cargo rustc. So I found the another way to compile with osx: cargo rustc -- -C link-args="-estart -nostartfiles -static" -C link-dead-code. I don't know why ,clang does mangle function _start to __start. the entrypoint function in main.rs is _start not start,but the linker args must be __start.

[phil-opp]

@mfournial Sorry for the late reply!

Adding the compiler flags in a .cargo/config in the root of the project like so:

[build]
rustflags = ["-C", "link-arg=-nostartfiles"]

keeps the build commands consistent accross OSes (cargo build)

So we don't need to pass -lSystem on macOS?

[phil-opp]

@reesmanp I also thought about adding conditional compilation and a .cargo/config to keep the build command simple and platform independent. I decided against it because we no longer need these arguments when we compile for a custom target in the next post. I'll add a short notice to the post.

[phil-opp]

After rereading all the feedback in this thread, I decided to rewrite the section about overwriting the entry point. It now uses a different approach to fixing the linker errors: compile for a bare metal target. The existing section about the linker arguments still exists, but is now optional and hidden by default. I updated the arguments so that now _start is used on all OSs and also explained how to create a .cargo/config file. For more details see the pull request at https://github.com/phil-opp/blog_os/pull/577.

Thank you all for your feedback and helping in improving this post!

[cstenger]

@phil-opp Thank you for all the hard work you've put into this blog. It's an amazing resource.

If anyone has issues completing the Linker Arguments portion on a Mac the following .cargo/config file worked for me (not tested on Linux and Windows yet).

[target.'cfg(target_os = "linux")']
rustflags = [
  "-C", "link-arg=-nostartfiles",
]

[target.'cfg(target_os = "windows")']
rustflags = [
  "-C", "link-arg=/ENTRY:_start",
  "-C", "link-arg=/SUBSYSTEM:console",
]

[target.'cfg(target_os = "macos")']
rustflags = [
  "-C", "link-arg=-lSystem",
  "-C", "link-arg=-e __start",
  "-C", "link-arg=-static",
  "-C", "link-arg=-nostartfiles",
]

The first change I made was separating the link-args into multiple link-arg arguments as it didn't seem to be parsing the spaces correctly. Then I added -lSystem to avoid the clang linker error ld: library not found for -lSystem.

[phil-opp]

@cstenger Thanks a lot :).

Interesting. I updated this section quite recently. I don't have a mac myself, so I was only able to test it on CI, where the command cargo rustc -- -C link-args="-e __start -static -nostartfiles" works fine on macOS. Does this command work for you, i.e. does the problem only occur with the cargo/.config file?

[cstenger]

Well, guess I'm going crazy. To ensure it wasn't due to any changes I had in my existing project, I started from scratch today. Everything worked as it should have. When calling rustc directly as well as from a config file, no "-lSystem" or multiple link-arg statements were required. Then I went back to the retrofitted project (where I encountered the issues), and it all worked there too. At this moment I don't know why the behavior doesn't match from the previous evening, if I'm able to reproduce it I'll be sure to document the circumstances. Hopefully my experiences haven't lead anyone astray.

[phil-opp]

@cstenger Thanks for the update! Let me know if you can reproduce the failure.

It's not that there's something wrong with -lSystem (it works too), I just want that the instructions on the blog work for everyone.

[ghost]

@phil-opp I followed each instruction here but I this happened:

C:\Users\Administrator\LunarOS>xargo +nightly build --target thumbv7em-none-eabihf --release --verbose
+ "rustc" "--print" "sysroot"
WARNING: the sysroot can't be built for the Stable channel. Switch to nightly.
+ "cargo" "+nightly" "build" "--target" "thumbv7em-none-eabihf" "--release" "--verbose"
   Compiling LunarOS v0.1.0 (C:\Users\Administrator\LunarOS)
     Running `rustc --edition=2018 --crate-name LunarOS src\main.rs --color always --crate-type bin --emit=dep-info,link -C opt-level=3 -C panic=abort -C metadata=b20ba269d00b8257 -C extra-filename=-b20ba269d00b8257 --out-dir C:\Users\Administrator\LunarOS\target\thumbv7em-none-eabihf\release\deps --target thumbv7em-none-eabihf -L dependency=C:\Users\Administrator\LunarOS\target\thumbv7em-none-eabihf\release\deps -L dependency=C:\Users\Administrator\LunarOS\target\release\deps`
error[E0463]: can't find crate for `core`

error: aborting due to previous error

For more information about this error, try `rustc --explain E0463`.
error: Could not compile `LunarOS`.

Caused by:
  process didn't exit successfully: `rustc --edition=2018 --crate-name LunarOS src\main.rs --color always --crate-type bin --emit=dep-info,link -C opt-level=3 -C panic=abort -C metadata=b20ba269d00b8257 -C extra-filename=-b20ba269d00b8257 --out-dir C:\Users\Administrator\LunarOS\target\thumbv7em-none-eabihf\release\deps --target thumbv7em-none-eabihf -L dependency=C:\Users\Administrator\LunarOS\target\thumbv7em-none-eabihf\release\deps -L dependency=C:\Users\Administrator\LunarOS\target\release\deps` (exit code: 1)

How can I fix this?

System Info:

OS Name Microsoft Windows 10 Home Version 10.0.17134 Build 17134 Other OS Description Not Available OS Manufacturer Microsoft Corporation System Name LAPTOP-FHA479JD System Manufacturer HP System Model OMEN by HP Laptop 15-ce0xx System Type x64-based PC System SKU 1KV79UA#ABA Processor Intel(R) Core(TM) i7-7700HQ CPU @ 2.80GHz, 2808 Mhz, 4 Core(s), 8 Logical Processor(s) BIOS Version/Date American Megatrends Inc. F.10, 11/1/2017 SMBIOS Version 3.0 Embedded Controller Version 40.24 BIOS Mode UEFI BaseBoard Manufacturer HP BaseBoard Model Not Available BaseBoard Name Base Board Platform Role Mobile Secure Boot State On PCR7 Configuration Binding Possible Windows Directory C:\WINDOWS System Directory C:\WINDOWS\system32 Boot Device \Device\HarddiskVolume1 Locale United States Hardware Abstraction Layer Version = "10.0.17134.765" User Name LAPTOP-FHA479JD\Administrator Time Zone Eastern Daylight Time Installed Physical Memory (RAM) 8.00 GB Total Physical Memory 7.88 GB Available Physical Memory 2.66 GB Total Virtual Memory 12.9 GB Available Virtual Memory 5.42 GB Page File Space 5.00 GB Page File C:\pagefile.sys Kernel DMA Protection Off Virtualization-based security Not enabled Device Encryption Support Reasons for failed automatic device encryption: Hardware Security Test Interface failed and device is not InstantGo, Un-allowed DMA capable bus/device(s) detected Hyper-V - VM Monitor Mode Extensions Yes Hyper-V - Second Level Address Translation Extensions Yes Hyper-V - Virtualization Enabled in Firmware Yes Hyper-V - Data Execution Protection Yes

Edit(phil-opp): Cleaned up formatting.

[ghost]

but this happened*

[phil-opp]

@LiamTheProgrammer Is there a reason why you're using xargo instead of cargo-xbuild?

The problem seems to be that xargo does not use the +nightly argument when using the sysroot. As a result, it tries to build the sysroot with the default stable Rust compiler, which results in this warning:

WARNING: the sysroot can't be built for the Stable channel. Switch to nightly.

To fix this, you can either add a nightly override for the LunarOS directory using rustup override add nightly or create a rust-toolchain file with the content "nightly` in that folder.

[cqh963852]

I am doing translation.And I have some doubts. Can I think of the normal entry point chain as the normal entry point

[bjorn3]

@TheBegining Normally there is a #[lang = "start"] function defined in libstd which gets main as argument. This function first performs some initialization like setting up a stack guard and storing the arguments of the process at a specific location. It then calls main. By using #![no_main] there doesn't need to be a #[lang = "start"] function, instead you have to define the entrypoint of the executable yourself.

[Fibration]

@phil-opp Thanks for the tutorial. I have been enjoying following it so far. I am stuck at cargo xbuild and have checked against your files in the post-02 branch to make sure that everything is identical.

I am receiving the following error at the end of cargo xbuild:

error: cannot produce bin for `os v0.1.0` as the target `x86_64-os` does not support these crate types

Cargo now uses a more informative error message. Issue here. I assume that xbuild is still using an older version of cargo.

I could not get any further than that. Any insight you have would be great.

[phil-opp]

@Fibration

Cargo now uses a more informative error message. Issue here. I assume that xbuild is still using an older version of cargo.

cargo-xbuild just calls out to the cargo you have installed. Maybe you did not add a nightly override for your directory (either though rustup override add or a rust-toolchain file) so that the default cargo is a stable version instead of a more recent nightly version?

Either way, the old error message contains the same number of information, it just uses a different formulation. The error indicates that your target does not support the bin crate type, which likely means that you don't have the "executables": true line in your target JSON: https://github.com/phil-opp/blog_os/blob/fea4fa9c6790c0bd234f1031d7a775e570c5d607/x86_64-blog_os.json#L9

Note that needs to be executables (plural), not executable. Unfortunately, cargo just ignores invalid target configuration keys instead of throwing an error for them.

I hope this solves your issue!

[Lokathor]

Infinite loop with no side effect is UB because of LLVM problems.

there is a -Z flag to fix this, https://github.com/rust-lang/rust/pull/59546

Or you can just volatile read a local variable over and over (or any other side effect of of choice).

[jerryajay]

Thank you.

[phil-opp]

@Lokathor Thanks for your comment! I'm aware of that issue. Everytime I look at it it seems like it is close to being solved, so I always thinks that it's not worth it to mention it on the blog. However, since the issue now persists for multiple years, you're probably right that we should fix it on the blog too. I opened https://github.com/phil-opp/blog_os/issues/793 to track this.

[phil-opp]

@jerryajay You're welcome!

[briankung]

Our freestanding executable does not have access to the Rust runtime and crt0, so we need to define our own entry point. Implementing the start language item wouldn't help, since it would still require crt0. Instead, we need to overwrite the crt0 entry point directly...The reason for naming the function _start is that this is the default entry point name for most systems.

What system or systems are responsible for calling the _start entrypoint?

[Lokathor]

since this booted to, i presume the linker is configured by default to arrange the binary for the _start to be at the initial address that the bootloader jumps to.

[briankung]

Thank you!