Closed JustAnotherCodemonkey closed 3 months ago
Sorry for the delay in my response. This looks very interesting, but I must apologize -- what exactly were you trying to do when you encountered this issue? I can't reproduce it even when I check out a fresh copy of Theseus on a new machine.
Perhaps were you trying to update Theseus to use the latest nightly compiler version? If so, yes, that generally requires a lot of effort, since the Rust team routinely makes substantial changes to the language in nightlies. Was there something else in particular that you were hoping to accomplish? perhaps I can help with that.
Sorry if I was not clear enough. Theseus looked interesting and I was looking to explore it, starting with the instructions in the quick start. I believe I now know the ultimate cause of the issue.
The instructions are great for most people, especially new to Rust but they're not very explicit on the why of what to do for people who are experienced in Rust and feel like they know what they are doing. I was unaware that Theseus relied on a specific version of nightly and didn't make the connection that it was automatically installing that correct version of nightly. Maybe this should be more clear or am I just very silly? I don't know.
I use NixOS where rustup is not used but instead, rustc, cargo, and rustfmt are installed together independently. To get nightly, I use oxalica's rust-overlay (https://github.com/oxalica/rust-overlay) which now that I research seems to be able to generate a Rust toolchain from a toolchain file such as I now see you have although that functionality is not immediately obvious.
I may be able to make a nix development shell out of Theseus and even put it in a PR now though (hopefully) and hopefully that would not need to change in the future so long as the toolchain file is not moved or removed. I may even be able to do some derivation stuff with it but no guarantees as I am obviously still a little green to nix and depending on what the derivation would be able to do, more or less of it would need changing when significant changes to the build process / makefile front-facing interface are made.
Hopefully the above is coherent as it is very late as I write this.
Update: I have successfully built and run Theseus so I'll likely close but still wondering about whether I should add the Nix stuff or documentation in a PR.
Oh I see, yeah we use the standard Rust-specific way of specifying the required nightly version, the rust-toolchain
file. Having never used Nix OS before, I'm curious how things work -- does it have its own version of Rust that is required? Or can it not work with the rust-toolchain
file somehow?
The instructions are great for most people, especially new to Rust but they're not very explicit on the why of what to do for people who are experienced in Rust and feel like they know what they are doing.
Ah, interesting observation. What else do you think we should add in the README to improve this scenario?
Update: I have successfully built and run Theseus so I'll likely close but still wondering about whether I should add the Nix stuff or documentation in a PR.
Awesome! Yes, please do, that would be great, especially since I have no experience with Nix.
Ok doing some research on how the build system works (I'm very used to the language but not rustup itself), it's interesting to me that cargo fails silently when a toolchain file is supplied but rustup is absent. This may be an issue with a Nix patch.
As for about Nix itself, the way it works is that everything happens through Nix itself as a package manager. The whole point of Nix is declarative and reproducible builds so you can see why having something like rustup around and circumventing Nix by installing your own toolchain externally would be frowned on as it could mess with other stuff and lead to impurity sneaking in. Therefore, Nix/Rust users will install rustc, cargo, rustfmt, and other tools in the toolchain separately through Nix itself.
Now to use nightly, many people use oxalica's "rust-overlay" (https://github.com/oxalica/rust-overlay/tree/master) which is an "overlay" which is sort of like a mod for the package source. This overlay supplies a way to specify a toolchain and easily use that. The issue comes in when you just specify "latest nightly please" and try to use that as the toolchain will be for the latest nightly build and it apparently ignore when a project requests a toolchain (likely because it cannot install a new toolchain itself by design, and decides to try to build the crate anyways and hope it works.) oxalica/rust-overlay is nice because it lets you build a toolchain from a file as well.
The solution would be to add a shell.nix, and it would be nice to make a flake for it too with even maybe a derivation. Essentially, a "pre-make" file that would set up the shell for building the project, including bringing in all necessary dependencies. It's pretty sweet but obviously can cause a lot of headaches for Nix noobs.
I'll see what I can do about creating all 3 of those things and will make a PR although I can only promise that first one, a shell setup file.
I should add just as a note that if you consider adding engaging the shell to the makefile, it's generally better to let people do that themselves and let Nix act as a wrapper on top of other build systems.
Steps to Reproduce
(with latest rustc nightly)
git clone --recurse-submodules --depth 1 https://github.com/theseus-os/Theseus.git cd Theseus make run
Relevant Output
=================== BUILDING ALL CRATES =================== TARGET: "x86_64-unknown-theseus" KERNEL_PREFIX: "k#" APP_PREFIX: "a#" CFLAGS: "" THESEUS_CONFIG (before build.rs script): "" THESEUS_CFLAGS='' THESEUS_NANO_CORE_BUILD_DIR='/home/sophiaw/Theseus/build/nano_core' RUST_TARGET_PATH='/home/sophiaw/Theseus/cfg' RUSTFLAGS='--emit=obj -C debuginfo=2 -D unused-must-use -Z share-generics=no ' cargo build --release --workspace --features nano_core/bios -Z unstable-options -Z build-std=core,alloc -Z build-std-features=compiler-builtins-mem --target x86_64-unknown-theseus [building crates] Compiling rustc-std-workspace-core v1.99.0 (/nix/store/f7affgwx75sffflvlmgiz5vi9w3vlvhb-rust-default-1.81.0-nightly-2024-06-22/lib/rustlib/src/rust/library/rustc-std-workspace-core) error: data-layout for target `x86_64-unknown-theseus`, `e-m:e-p270:32:32-p271:32:32-p272:64:64-i64:64-f80:128-n8:16:32:64-S128`, differs from LLVM target's `x86_64-unknown-none-elf` default layout, `e-m:e-p270:32:32-p271:32:32-p272:64:64-i64:64-i128:128-f80:128-n8:16:32:64-S128` error: could not compile `rustc-std-workspace-core` (lib) due to 1 previous error warning: build failed, waiting for other jobs to finish... error: could not compile `core` (lib) due to 1 previous error error: could not compile `compiler_builtins` (lib) due to 1 previous error make: *** [Makefile:330: cargo] Error 101
Attempted Mitigation and Following Errors
Patch cfg/x86_64-unknown-theseus.json:
[line 5] "data-layout": "e-m:e-p270:32:32-p271:32:32-p272:64:64-i64:64-f80:128-n8:16:32:64-S128",
->"data-layout": "e-m:e-p270:32:32-p271:32:32-p272:64:64-i64:64-i128:128-f80:128-n8:16:32:64-S128",
- New error:
error[E0635]: unknown feature `stdsimd` --> /home/sophiaw/.cargo/registry/src/index.crates.io-6f17d22bba15001f/ahash-0.7.6/src/lib.rs:33:42 | 33 | #![cfg_attr(feature = "stdsimd", feature(stdsimd))] | ^^^^^^^ [...] For more information about this error, try `rustc --explain E0635`. error: could not compile `ahash` (lib) due to 1 previous error warning: build failed, waiting for other jobs to finish... make: *** [Makefile:330: cargo] Error 101
Update ahash to patch for this error using
cargo update ahash
- Relevant output containing errors and warnings:
[...] warning: the feature `ptr_internals` is internal to the compiler or standard library --> kernel/vga_buffer/src/lib.rs:6:12 | 6 | #![feature(ptr_internals)] | ^^^^^^^^^^^^^ | = note: using it is strongly discouraged = note: `#[warn(internal_features)]` on by default Compiling color v0.1.0 (/home/sophiaw/Theseus/kernel/color) warning: field `0` is never read --> kernel/vga_buffer/src/lib.rs:120:22 | 120 | pub struct ColorCode(u8); | --------- ^^ | | | field in this struct | = help: consider removing this field = note: `ColorCode` has derived impls for the traits `Clone` and `Debug`, but these are intentionally ignored during dead code analysis = note: `#[warn(dead_code)]` on by default Compiling context_switch v0.1.0 (/home/sophiaw/Theseus/kernel/context_switch) Compiling logger v0.1.0 (/home/sophiaw/Theseus/kernel/logger) Compiling rand_core v0.6.3 warning: unexpected `cfg` condition name: `simd_personality` --> kernel/context_switch/src/lib.rs:13:14 | 13 | if #[cfg(simd_personality)] { | ^^^^^^^^^^^^^^^^ | = help: expected names are: `clippy`, `debug_assertions`, `doc`, `docsrs`, `doctest`, `feature`, `miri`, `overflow_checks`, `panic`, `proc_macro`, `relocation_model`, `rustfmt`, `sanitize`, `sanitizer_cfi_generalize_pointers`, `sanitizer_cfi_normalize_integers`, `target_abi`, `target_arch`, `target_endian`, `target_env`, `target_family`, `target_feature`, `target_has_atomic`, `target_has_atomic_equal_alignment`, `target_has_atomic_load_store`, `target_os`, `target_pointer_width`, `target_thread_local`, `target_vendor`, `test`, `ub_checks`, `unix`, and `windows` = help: consider using a Cargo feature instead = help: or consider adding in `Cargo.toml` the `check-cfg` lint config for the lint: [lints.rust] unexpected_cfgs = { level = "warn", check-cfg = ['cfg(simd_personality)'] } = help: or consider adding `println!("cargo::rustc-check-cfg=cfg(simd_personality)");` to the top of the `build.rs` = note: see <https://doc.rust-lang.org/nightly/rustc/check-cfg/cargo-specifics.html> for more information about checking conditional configuration = note: `#[warn(unexpected_cfgs)]` on by default Compiling mouse_data v0.1.0 (/home/sophiaw/Theseus/libs/mouse_data) Compiling hash32 v0.2.1 warning: unexpected `cfg` condition name: `mirror_log_to_vga` --> kernel/logger/src/lib.rs:28:7 | 28 | #[cfg(mirror_log_to_vga)] | ^^^^^^^^^^^^^^^^^ | = help: expected names are: `clippy`, `debug_assertions`, `doc`, `docsrs`, `doctest`, `feature`, `miri`, `overflow_checks`, `panic`, `proc_macro`, `relocation_model`, `rustfmt`, `sanitize`, `sanitizer_cfi_generalize_pointers`, `sanitizer_cfi_normalize_integers`, `target_abi`, `target_arch`, `target_endian`, `target_env`, `target_family`, `target_feature`, `target_has_atomic`, `target_has_atomic_equal_alignment`, `target_has_atomic_load_store`, `target_os`, `target_pointer_width`, `target_thread_local`, `target_vendor`, `test`, `ub_checks`, `unix`, and `windows` = help: consider using a Cargo feature instead = help: or consider adding in `Cargo.toml` the `check-cfg` lint config for the lint: [lints.rust] unexpected_cfgs = { level = "warn", check-cfg = ['cfg(mirror_log_to_vga)'] } = help: or consider adding `println!("cargo::rustc-check-cfg=cfg(mirror_log_to_vga)");` to the top of the `build.rs` = note: see <https://doc.rust-lang.org/nightly/rustc/check-cfg/cargo-specifics.html> for more information about checking conditional configuration = note: `#[warn(unexpected_cfgs)]` on by default warning: unexpected `cfg` condition name: `mirror_log_to_vga` --> kernel/logger/src/lib.rs:392:7 | 392 | #[cfg(mirror_log_to_vga)] | ^^^^^^^^^^^^^^^^^ | = help: consider using a Cargo feature instead = help: or consider adding in `Cargo.toml` the `check-cfg` lint config for the lint: [lints.rust] unexpected_cfgs = { level = "warn", check-cfg = ['cfg(mirror_log_to_vga)'] } = help: or consider adding `println!("cargo::rustc-check-cfg=cfg(mirror_log_to_vga)");` to the top of the `build.rs` = note: see <https://doc.rust-lang.org/nightly/rustc/check-cfg/cargo-specifics.html> for more information about checking conditional configuration warning: unexpected `cfg` condition name: `mirror_log_to_vga` --> kernel/logger/src/lib.rs:257:15 | 257 | #[cfg(mirror_log_to_vga)] | ^^^^^^^^^^^^^^^^^ | = help: consider using a Cargo feature instead = help: or consider adding in `Cargo.toml` the `check-cfg` lint config for the lint: [lints.rust] unexpected_cfgs = { level = "warn", check-cfg = ['cfg(mirror_log_to_vga)'] } = help: or consider adding `println!("cargo::rustc-check-cfg=cfg(mirror_log_to_vga)");` to the top of the `build.rs` = note: see <https://doc.rust-lang.org/nightly/rustc/check-cfg/cargo-specifics.html> for more information about checking conditional configuration warning: `context_switch` (lib) generated 1 warning Compiling ppv-lite86 v0.2.17 Compiling boot_info v0.1.0 (/home/sophiaw/Theseus/kernel/boot_info) warning: `vga_buffer` (lib) generated 2 warnings Compiling frame_allocator v0.1.0 (/home/sophiaw/Theseus/kernel/frame_allocator) Compiling page_allocator v0.1.0 (/home/sophiaw/Theseus/kernel/page_allocator) Compiling bootloader_modules v0.1.0 (/home/sophiaw/Theseus/kernel/bootloader_modules) Compiling stable_deref_trait v1.2.0 Compiling rdrand v0.8.2 Compiling tsc v0.1.0 (/home/sophiaw/Theseus/kernel/tsc) error[E0308]: mismatched types --> kernel/boot_info/src/multiboot2.rs:171:20 | 21 | type MemoryRegionIterator<'a> = impl Iterator<Item = &'a multiboot2::MemoryArea>; | ------------------------------------------------ the expected opaque type ... 171 | inner: self | ____________________^ 172 | | .memory_map_tag() 173 | | .ok_or("no memory map tag")? 174 | | .memory_areas(), | |_______________________________^ expected opaque type, found a different opaque type | ::: /home/sophiaw/.cargo/registry/src/index.crates.io-6f17d22bba15001f/multiboot2-0.14.0/src/memory_map.rs:26:35 | 26 | pub fn memory_areas(&self) -> impl Iterator<Item = &MemoryArea> { | --------------------------------- the found opaque type | = note: expected opaque type `MemoryRegionIterator<'_>` found opaque type `impl Iterator<Item = &MemoryArea>` = note: distinct uses of `impl Trait` result in different opaque types Compiling managed v0.8.0 Compiling rand v0.8.5 Compiling downcast-rs v1.2.0 Compiling stdio v0.1.0 (/home/sophiaw/Theseus/libs/stdio) Compiling event_types v0.1.0 (/home/sophiaw/Theseus/kernel/event_types) For more information about this error, try `rustc --explain E0308`. error: could not compile `boot_info` (lib) due to 1 previous error warning: build failed, waiting for other jobs to finish... warning: `logger` (lib) generated 3 warnings warning: fields `0` and `1` are never read --> kernel/frame_allocator/src/lib.rs:875:20 | 875 | AddressNotFree(Frame<Page4K>, usize), | -------------- ^^^^^^^^^^^^^ ^^^^^ | | | fields in this variant | = note: `#[warn(dead_code)]` on by default help: consider changing the fields to be of unit type to suppress this warning while preserving the field numbering, or remove the fields | 875 | AddressNotFree((), ()), | ~~ ~~ warning: fields `0` and `1` are never read --> kernel/frame_allocator/src/lib.rs:877:21 | 877 | AddressNotFound(Frame<Page4K>, usize), | --------------- ^^^^^^^^^^^^^ ^^^^^ | | | fields in this variant | help: consider changing the fields to be of unit type to suppress this warning while preserving the field numbering, or remove the fields | 877 | AddressNotFound((), ()), | ~~ ~~ warning: field `0` is never read --> kernel/frame_allocator/src/lib.rs:880:23 | 880 | OutOfAddressSpace(usize), | ----------------- ^^^^^ | | | field in this variant | help: consider changing the field to be of unit type to suppress this warning while preserving the field numbering, or remove the field | 880 | OutOfAddressSpace(()), | ~~ warning: `frame_allocator` (lib) generated 3 warnings make: *** [Makefile:330: cargo] Error 101
Conclusion
At this point, I gave up. I hope this was the nicest issue you've ever read though. Project looks very impressive and I'd love to see it when patches come out for the above errors.
Sincerely, JustAnotherCodemokey
same error but upon cargo build
rustc --explain E0308
Expected type did not match the received type.
error[E0308]: mismatched types --> kernel/boot_info/src/multiboot2.rs:171:20 | 21 | type MemoryRegionIterator<'a> = impl Iterator<Item = &'a multiboot2::MemoryArea>; | ------------------------------------------------ the expected opaque type ... 171 | inner: self | ____________________^ 172 | | .memory_map_tag() 173 | | .ok_or("no memory map tag")? 174 | | .memory_areas(), | |_______________________________^ expected opaque type, found a different opaque type | ::: /home/dev/.cargo/registry/src/index.crates.io-6f17d22bba15001f/multiboot2-0.14.2/src/memory_map.rs:26:35 | 26 | pub fn memory_areas(&self) -> impl Iterator<Item = &MemoryArea> { | --------------------------------- the found opaque type | = note: expected opaque type `MemoryRegionIterator<'_>` found opaque type `impl Iterator<Item = &MemoryArea>` = note: distinct uses of `impl Trait` result in different opaque types
warning: fields 0 and 1 are never read
--> kernel/frame_allocator/src/lib.rs:875:20 |
875 | AddressNotFree(Frame |
-------------- ^^^^^^^^^^^^^ ^^^^^ |
---|---|---|---|
fields in this variant | |||
= note: `AllocationError` has a derived impl for the trait `Debug`, but this is intentionally ignored during dead code analysis
= note: `#[warn(dead_code)]` on by default
help: consider changing the fields to be of unit type to suppress this warning while preserving the field numbering, or remove the fields | 875 | AddressNotFree((), ()), | ~~ ~~
warning: fields 0 and 1 are never read
--> kernel/frame_allocator/src/lib.rs:877:21 |
877 | AddressNotFound(Frame |
--------------- ^^^^^^^^^^^^^ ^^^^^ |
---|---|---|---|
fields in this variant | |||
= note: `AllocationError` has a derived impl for the trait `Debug`, but this is intentionally ignored during dead code analysis
help: consider changing the fields to be of unit type to suppress this warning while preserving the field numbering, or remove the fields | 877 | AddressNotFound((), ()), | ~~ ~~
warning: field 0 is never read
--> kernel/frame_allocator/src/lib.rs:880:23 |
880 | OutOfAddressSpace(usize), | ----------------- ^^^^^ |
---|---|---|---|
field in this variant | |||
= note: `AllocationError` has a derived impl for the trait `Debug`, but this is intentionally ignored during dead code analysis
help: consider changing the field to be of unit type to suppress this warning while preserving the field numbering, or remove the field | 880 | OutOfAddressSpace(()), | ~~
exact function
fn memory_regions(&self) -> Result<Self::MemoryRegions<'_>, &'static str> {
Ok(MemoryRegions {
inner: self
.memory_map_tag()
.ok_or("no memory map tag")?
.memory_areas(),
})
}
Steps to Reproduce
(with latest rustc nightly)
Relevant Output
Attempted Mitigation and Following Errors
[line 5] "data-layout": "e-m:e-p270:32:32-p271:32:32-p272:64:64-i64:64-f80:128-n8:16:32:64-S128",
->"data-layout": "e-m:e-p270:32:32-p271:32:32-p272:64:64-i64:64-i128:128-f80:128-n8:16:32:64-S128",
[...]
For more information about this error, try
rustc --explain E0635
. error: could not compileahash
(lib) due to 1 previous error warning: build failed, waiting for other jobs to finish... make: *** [Makefile:330: cargo] Error 101[...]
ptr_internals
is internal to the compiler or standard library --> kernel/vga_buffer/src/lib.rs:6:12= note: using it is strongly discouraged = note:
#[warn(internal_features)]
on by default0
is never read --> kernel/vga_buffer/src/lib.rs:120:22cfg
condition name:simd_personality
--> kernel/context_switch/src/lib.rs:13:14= help: expected names are:
clippy
,debug_assertions
,doc
,docsrs
,doctest
,feature
,miri
,overflow_checks
,panic
,proc_macro
,relocation_model
,rustfmt
,sanitize
,sanitizer_cfi_generalize_pointers
,sanitizer_cfi_normalize_integers
,target_abi
,target_arch
,target_endian
,target_env
,target_family
,target_feature
,target_has_atomic
,target_has_atomic_equal_alignment
,target_has_atomic_load_store
,target_os
,target_pointer_width
,target_thread_local
,target_vendor
,test
,ub_checks
,unix
, andwindows
= help: consider using a Cargo feature instead = help: or consider adding inCargo.toml
thecheck-cfg
lint config for the lint: [lints.rust] unexpected_cfgs = { level = "warn", check-cfg = ['cfg(simd_personality)'] } = help: or consider addingprintln!("cargo::rustc-check-cfg=cfg(simd_personality)");
to the top of thebuild.rs
= note: see https://doc.rust-lang.org/nightly/rustc/check-cfg/cargo-specifics.html for more information about checking conditional configuration = note:#[warn(unexpected_cfgs)]
on by defaultcfg
condition name:mirror_log_to_vga
--> kernel/logger/src/lib.rs:28:7= help: expected names are:
clippy
,debug_assertions
,doc
,docsrs
,doctest
,feature
,miri
,overflow_checks
,panic
,proc_macro
,relocation_model
,rustfmt
,sanitize
,sanitizer_cfi_generalize_pointers
,sanitizer_cfi_normalize_integers
,target_abi
,target_arch
,target_endian
,target_env
,target_family
,target_feature
,target_has_atomic
,target_has_atomic_equal_alignment
,target_has_atomic_load_store
,target_os
,target_pointer_width
,target_thread_local
,target_vendor
,test
,ub_checks
,unix
, andwindows
= help: consider using a Cargo feature instead = help: or consider adding inCargo.toml
thecheck-cfg
lint config for the lint: [lints.rust] unexpected_cfgs = { level = "warn", check-cfg = ['cfg(mirror_log_to_vga)'] } = help: or consider addingprintln!("cargo::rustc-check-cfg=cfg(mirror_log_to_vga)");
to the top of thebuild.rs
= note: see https://doc.rust-lang.org/nightly/rustc/check-cfg/cargo-specifics.html for more information about checking conditional configuration = note:#[warn(unexpected_cfgs)]
on by defaultcfg
condition name:mirror_log_to_vga
--> kernel/logger/src/lib.rs:392:7cfg
condition name:mirror_log_to_vga
--> kernel/logger/src/lib.rs:257:15context_switch
(lib) generated 1 warning Compiling ppv-lite86 v0.2.17 Compiling boot_info v0.1.0 (/home/sophiaw/Theseus/kernel/boot_info) warning:vga_buffer
(lib) generated 2 warnings Compiling frame_allocator v0.1.0 (/home/sophiaw/Theseus/kernel/frame_allocator) Compiling page_allocator v0.1.0 (/home/sophiaw/Theseus/kernel/page_allocator) Compiling bootloader_modules v0.1.0 (/home/sophiaw/Theseus/kernel/bootloader_modules) Compiling stable_deref_trait v1.2.0 Compiling rdrand v0.8.2 Compiling tsc v0.1.0 (/home/sophiaw/Theseus/kernel/tsc) error[E0308]: mismatched types --> kernel/boot_info/src/multiboot2.rs:171:20rustc --explain E0308
. error: could not compileboot_info
(lib) due to 1 previous error warning: build failed, waiting for other jobs to finish... warning:logger
(lib) generated 3 warnings warning: fields0
and1
are never read --> kernel/frame_allocator/src/lib.rs:875:20help: consider changing the fields to be of unit type to suppress this warning while preserving the field numbering, or remove the fields | 875 | AddressNotFree((), ()), | ~~ ~~
0
and1
are never read --> kernel/frame_allocator/src/lib.rs:877:21help: consider changing the fields to be of unit type to suppress this warning while preserving the field numbering, or remove the fields | 877 | AddressNotFound((), ()), | ~~ ~~
0
is never read --> kernel/frame_allocator/src/lib.rs:880:23help: consider changing the field to be of unit type to suppress this warning while preserving the field numbering, or remove the field | 880 | OutOfAddressSpace(()), | ~~
warning:
frame_allocator
(lib) generated 3 warnings make: *** [Makefile:330: cargo] Error 101