includeos / IncludeOS

A minimal, resource efficient unikernel for cloud services
https://www.includeos.org
Apache License 2.0
4.86k stars 358 forks source link

Nix static wip - part 2 #2231

Closed alfreb closed 1 month ago

alfreb commented 1 month ago

Thanks to @MagnusS IncludeOS now compiles with nix using musl 🎉

And, a static nix-build of LLVM's libc++. I'm looking into whether we can link against that as-is, without rolling our own build of LLVM at all.

While we build our own musl-includeos (disabled in the most recent commit) the musl from nixpkgs is the one in use during compilation for now. See for example, this issue, now fixed, where it references musl-static-x86_64-unknown-linux-musl-1.2.3, which is quite recent compared to the 1.1.18 musl-includeos is pinned to.

/build/source/src/crt/quick_exit.cpp:13:5: error: exception specification in declaration does not match previous declaration         
int at_quick_exit (void (*func)()) noexcept                                                                                          
    ^                                                                                                                                
/nix/store/6gc6xiawd9xcvscd6p0wcjcm8b42fmwh-musl-static-x86_64-unknown-linux-musl-1.2.3-dev/include/stdlib.h:50:5: note: previous de\
claration is here                                                                                                                    
int at_quick_exit (void (*) (void));                                                                                                 

So we should expect some incompatibilities with musl-includeos when we get to linking a bootable binary, where we absolutely have to have our own libc, because we need to patch in our own system calls.

In the best case we can compile IncludeOS itself against the existing musl package, and then only use our own when linking the bootable binary. This would make our build setup simpler and easier to maintain, avoiding the need for -nostdinc when compiling IncludeOS. Worst case we'll need to make our own nix build of libcxx, libunwind and friends, to compile them and the IncludeOS libs against musl-includeos.

Anyway, this setup is much closer to something working than what's in at the moment so if a clean build can be reproduced I think we should merge.

alfredb@uniperf-builder:~/IncludeOS$ tree result/lib/
result/lib/
├── libarch.a
├── libmusl_syscalls.a
└── libos.a

0 directories, 3 files
alfredb@uniperf-builder:~/IncludeOS$ tree result/drivers/
result/drivers/
├── libboot_logger.a
├── libe1000.a
├── libip4_reassembly.a
├── libtimestamps.a
├── libvga_emergency.a
├── libvirtioblk.a
├── libvirtiocon.a
├── libvirtionet.a
├── libvmxnet3.a
└── stdout
    ├── libdefault_stdout.a
    └── libvga_output.a

1 directory, 11 files
alfredb@uniperf-builder:~/IncludeOS$ tree result/platform/
result/platform/
└── libx86_64_pc.a

0 directories, 1 file
alfredb@uniperf-builder:~/IncludeOS$ tree result/plugins/
result/plugins/
├── libautoconf.a
├── libexample.a
├── libfield_medic.a
├── libmadness.a
├── libnacl.a
├── libsyslog.a
├── libsyslogd.a
├── libsystem_log.a
├── libterminal.a
├── libunik.a
└── libvfs.a
alfreb commented 1 month ago

The last commit unpins nixpkgs in order to get around an issue where clang "fails to compile a simple test program" according to cmake. It's apparently fixed in unstable, broken in both 23.11 and 24.05-pre. But I think this is the best setup we have so far.

alfreb commented 1 month ago

I figured out how to pin to a specific git revision now, so this now pins nixpkgs to the May 1st. commit we happened to have locally in our nixpkgs-unstable stream.

MagnusS commented 1 month ago

The additional commits LGTM. It's not complete yet, but we can continue to iterate in future PRs!