Closed ericcurtin closed 1 year ago
We gradually move to newer versions (and the goal is to eventually have a minimum version), but please note that the kernel only supports a single version at the moment.
It was working really well upto 1.66 with one or two small patches. I understand things are changing rapidly. Could be a good "first issue" for someone looking for an excuse to write Rust 🙂
It might have even just been one patch for 1.66, the allowlist/blocklist one...
My point was that it is expected that the kernel may not build with newer versions, i.e. it is a policy. In that sense, it does not matter how big the diff is or how fast things change. In other words, please do not expect the version to change only because a distribution upgraded their toolchain. The kernel will be upgrading the version gradually as needed for kernel purposes, but we are not version-aligned to a particular distribution or anything like that.
The kernel will be upgrading the version gradually as needed for kernel purposes, but we are not version-aligned to a particular distribution or anything like that.
I get where you're coming from here, but unlike C/C++ compilers, currently no Rust compiler implementation has multiple stable version streams supported in parallel. This creates a significant problem as we're left with choosing to deal with either an unsupported compiler or unable to build Rust components in the kernel.
Version aligning with multiple distros at the same time is impossible and picking one could be considered picking a favorite and upset other distros. We are slowly working towards removing the need for using unstable rustc features (https://github.com/Rust-for-Linux/linux/issues/2) either by having them stabilized (for example the bump from rust 1.62 to rust 1.66 stabilized most of the unstable features we allow in driver code: https://github.com/Rust-for-Linux/linux/commit/3f893e11f62f880f6d4e2bf6cda199a4366203a7#diff-feb504fc2c4587fa905005dc11712a69611fc2522ac3ea794250981667ce2522) or by replacing with alternatives. Until this is complete, rust for linux is tied to a specific rustc version. While it would technically be possible to support multiple rustc versions at the same time, this would require duplicating the vendored liballoc we have and sometimes unstable features change drastically enough that duplicating user code is the only option. It would be hard to keep both copies in sync.
I agree it isn't ideal. I'm trying to help reduce the amount of used unstable features to make it easier to build with newer rustc versions.
currently no Rust compiler implementation has multiple stable version streams supported in parallel.
By upstream Rust, perhaps, but there are other sources of support for toolchains, e.g. distributions.
This creates a significant problem as we're left with choosing to deal with either an unsupported compiler or unable to build Rust components in the kernel.
We understand, but "unsupported" in that sentence only means "by upstream Rust".
Moreover, please note that even if we track upstream Rust versions very closely, it would still be fairly problematic even for rolling distributions: there would still be delays/blocks of up to ~2 months where you cannot upgrade your Rust version to the latest until the next kernel releases.
So, in the end, whatever we do (in mainline, that is) will only work for a distribution/company that happens to align exactly with the (kernel, Rust) version tuples as they get released (until the kernel declares a minimum version). Either that, or the distribution/company is willing to support older compilers for a bit, or willing to backport new version support when it arrives in linux-next.
Of course, outside of mainline, it is always possible to look into providing something else on our own, and we have been considering such extra trees for distributions/companies.
Since I don't see it mentioned -- this particular error message can be solved by applying a change like the alloc
part of https://github.com/rust-lang/rust/commit/2786acce98a87a22a01fdc0ab82ca6f90d62d44e, specifically following the cfg(not(bootstrap))
mode (without having that cfg
yourself).
So, in the end, whatever we do (in mainline, that is) will only work for a distribution/company that happens to align exactly with the (kernel, Rust) version tuples as they get released (until the kernel declares a minimum version). Either that, or the distribution/company is willing to support older compilers for a bit, or willing to backport new version support when it arrives in linux-next.
I think it's a complete non-starter to port changes in unstable features to different compiler versions. Even a relatively simple change like the above commit would be way too invasive to apply on an older version or revert on a newer version. It seems much more feasible to patch the kernel's use of unstable features to match the system toolchain, even when that's a moving target. The fact that these are explicitly unstable also makes it clear who bears that responsibility, IMO, but I'm willing to help figure that out in the distros I maintain.
I am also interested in doing what I can to help stabilize things, or move away from unstable features where possible, but I'm guessing there's not much low-hanging fruit here.
I think it's a complete non-starter to port changes in unstable features to different compiler versions.
I think @ojeda meant to backport the changes to the code in linux-next for adapting to newer rustc version to whichever kernel version you are using rather than port changes in rustc itself to a different rustc. We try to keep usage of unstable features out of drivers as much as possible, so only changes in the rust/
directory and (the rust compilation section of) scripts/Makefile.build
should need to be backported and not any changes in actual drivers.
Great, then I think we're in alignment there, and it's nice to know that it should be relatively contained.
Indeed @bjorn3, I meant backporting the newer Rust support to an older kernel release.
For reference rust_allowed_features
in scripts/Makefile.build
lists all unstable features driver code is allowed to use. These are enabled by default and attempting to use #![feature]
to enable any other will result in an error thanks to -Zallow-features
. Currently only allocator_api
and const_refs_to_cell
are allowed. allocator_api
for the try_*
methods in our patched liballoc
. These will keep working unless you change rust/alloc
, so it is unlikely that you will get trouble with it in the near future. We use const_refs_to_cell
to make the implementation of offset_of!
and container_of!
work in const contexts. It has an outstanding concern, so breakage from this may occur, but I'm personally not too worried about it breaking any time soon. Once https://github.com/rust-lang/rust/pull/106934 lands we will probably replace it with the builtin offset_of!
from this PR.
Since I don't see it mentioned -- this particular error message can be solved by applying a change like the
alloc
part of rust-lang/rust@2786acc, specifically following thecfg(not(bootstrap))
mode (without having thatcfg
yourself).
Thanks for the pointer, I think I figured out some low hanging fruits last week:
https://github.com/ericcurtin/linux/commit/3eed43b20062c1867d509d6c8477618945684461 https://github.com/ericcurtin/linux/commit/24b5b8ebaff8db941d5616b5369efdd453963f7b
Then hit some other issues, I think it was ScopeGuard ones, if memory serves me right.
So, in the end, whatever we do (in mainline, that is) will only work for a distribution/company that happens to align exactly with the (kernel, Rust) version tuples as they get released (until the kernel declares a minimum version). Either that, or the distribution/company is willing to support older compilers for a bit, or willing to backport new version support when it arrives in linux-next.
So... We may submit Rust 1.67 changes to the rust or rust-next branch here? @ojeda @bjorn3
Or not at present?
Just so I understand the point correctly.
I think it's a complete non-starter to port changes in unstable features to different compiler versions. Even a relatively simple change like the above commit would be way too invasive to apply on an older version or revert on a newer version. It seems much more feasible to patch the kernel's use of unstable features to match the system toolchain, even when that's a moving target. The fact that these are explicitly unstable also makes it clear who bears that responsibility, IMO, but I'm willing to help figure that out in the distros I maintain.
I am also interested in doing what I can to help stabilize things, or move away from unstable features where possible, but I'm guessing there's not much low-hanging fruit here.
So... We may submit Rust 1.67 changes to the rust or rust-next branch here? @ojeda @bjorn3
No, please do not submit compiler upgrades. However, what would be really useful is to know:
Just so I understand the point correctly.
The part you quote is where I explained why updating the compiler version in the kernel does not really solve your issue since the kernel and the Rust versions cannot be always perfectly aligned due to different release schedules. Do you agree, or am I missing something? Or was something unclear?
What versions/policy would work best for you (or your distribution, if you speak for them).
Fedora follows upstream Rust's lifecycle closely. We upgrade the compiler across all stable releases every six weeks to the latest one.
Yes, I understand that is the policy for Rust upgrades in Fedora, but there are still conflicts you would need to solve even if the kernel released always with the latest Rust available version.
The question is what policy would you follow to resolve those. Would you block Rust upgrades until the next kernel release? Would you maintain two Rust toolchains at times? Would you patch your kernel backporting the newer support?
@ericcurtin
Thanks for the pointer, I think I figured out some low hanging fruits last week:
Adding #[cfg(bootstrap)]
will hide those things entirely, because that flag will not be set during your build. This is not the same thing as the RUSTC_BOOTSTRAP=1
environment hack.
When fixing kernel/rust/alloc
code that looks similar to rust/library/alloc
code, you'll want it to look more like the #[cfg(not(bootstrap))]
parts, but you don't need that cfg
attribute itself. For bespoke kernel code like ScopeGuard
, you'll need to understand the changes more directly, but I suspect you only ran into trouble there because you had cfg
'ed out the Fn
impls for Box
in your first commit.
@ojeda
The question is what policy would you follow to resolve those. Would you block Rust upgrades until the next kernel release? Would you maintain two Rust toolchains at times? Would you patch your kernel backporting the newer support?
Speaking only for the toolchain side, no I would not like to wait for the kernel, nor maintain multiple toolchains. I think patching the kernel with newer support will be best, and I expect that will be easier if those changes already exist somewhere to cherry-pick -- e.g. if some linux branch tracks stable or even beta Rust.
FWIW, with Fedora's rustc 1.67.1 and bindgen 0.63.0, this was enough to let me build:
diff --git a/rust/Makefile b/rust/Makefile
index 8f598a904f38..066f8e48d248 100644
--- a/rust/Makefile
+++ b/rust/Makefile
@@ -333,8 +333,8 @@ $(obj)/bindings/bindings_generated.rs: $(src)/bindings/bindings_helper.h \
# given it is `libclang`; but for consistency, future Clang changes and/or
# a potential future GCC backend for `bindgen`, we disable it too.
$(obj)/bindings/bindings_helpers_generated.rs: private bindgen_target_flags = \
- --blacklist-type '.*' --whitelist-var '' \
- --whitelist-function 'rust_helper_.*'
+ --blocklist-type '.*' --allowlist-var '' \
+ --allowlist-function 'rust_helper_.*'
$(obj)/bindings/bindings_helpers_generated.rs: private bindgen_target_cflags = \
-I$(objtree)/$(obj) -Wno-missing-prototypes -Wno-missing-declarations
$(obj)/bindings/bindings_helpers_generated.rs: private bindgen_target_extra = ; \
diff --git a/rust/alloc/boxed.rs b/rust/alloc/boxed.rs
index 7875adbab839..4f42cc5f9ad2 100644
--- a/rust/alloc/boxed.rs
+++ b/rust/alloc/boxed.rs
@@ -160,6 +160,7 @@ use core::hash::{Hash, Hasher};
#[cfg(not(no_global_oom_handling))]
use core::iter::FromIterator;
use core::iter::{FusedIterator, Iterator};
+use core::marker::Tuple;
use core::marker::{Destruct, Unpin, Unsize};
use core::mem;
use core::ops::{
@@ -1982,7 +1983,7 @@ impl<I: ExactSizeIterator + ?Sized, A: Allocator> ExactSizeIterator for Box<I, A
impl<I: FusedIterator + ?Sized, A: Allocator> FusedIterator for Box<I, A> {}
#[stable(feature = "boxed_closure_impls", since = "1.35.0")]
-impl<Args, F: FnOnce<Args> + ?Sized, A: Allocator> FnOnce<Args> for Box<F, A> {
+impl<Args: Tuple, F: FnOnce<Args> + ?Sized, A: Allocator> FnOnce<Args> for Box<F, A> {
type Output = <F as FnOnce<Args>>::Output;
extern "rust-call" fn call_once(self, args: Args) -> Self::Output {
@@ -1991,14 +1992,14 @@ impl<Args, F: FnOnce<Args> + ?Sized, A: Allocator> FnOnce<Args> for Box<F, A> {
}
#[stable(feature = "boxed_closure_impls", since = "1.35.0")]
-impl<Args, F: FnMut<Args> + ?Sized, A: Allocator> FnMut<Args> for Box<F, A> {
+impl<Args: Tuple, F: FnMut<Args> + ?Sized, A: Allocator> FnMut<Args> for Box<F, A> {
extern "rust-call" fn call_mut(&mut self, args: Args) -> Self::Output {
<F as FnMut<Args>>::call_mut(self, args)
}
}
#[stable(feature = "boxed_closure_impls", since = "1.35.0")]
-impl<Args, F: Fn<Args> + ?Sized, A: Allocator> Fn<Args> for Box<F, A> {
+impl<Args: Tuple, F: Fn<Args> + ?Sized, A: Allocator> Fn<Args> for Box<F, A> {
extern "rust-call" fn call(&self, args: Args) -> Self::Output {
<F as Fn<Args>>::call(self, args)
}
diff --git a/rust/alloc/lib.rs b/rust/alloc/lib.rs
index d361238272db..c4e7f80f6437 100644
--- a/rust/alloc/lib.rs
+++ b/rust/alloc/lib.rs
@@ -152,6 +152,7 @@
#![feature(trusted_len)]
#![feature(trusted_random_access)]
#![feature(try_trait_v2)]
+#![feature(tuple_trait)]
#![feature(unchecked_math)]
#![feature(unicode_internals)]
#![feature(unsize)]
diff --git a/rust/kernel/lib.rs b/rust/kernel/lib.rs
index c59947ac72ba..513a35a63d25 100644
--- a/rust/kernel/lib.rs
+++ b/rust/kernel/lib.rs
@@ -258,7 +258,4 @@ fn panic(info: &core::panic::PanicInfo<'_>) -> ! {
pr_emerg!("{}\n", info);
// SAFETY: FFI call.
unsafe { bindings::BUG() };
- // Bindgen currently does not recognize `__noreturn` so `BUG` returns `()`
- // instead of `!`. See <https://github.com/rust-lang/rust-bindgen/issues/2094>.
- loop {}
}
This also passed a build with rustup's 1.68.0-beta.6
, and then 1.69.0-nightly (7281249a1 2023-02-27)
needs to drop a feature that was just stabilized:
diff --git a/rust/alloc/lib.rs b/rust/alloc/lib.rs
index c4e7f80f6437..0338a3c91295 100644
--- a/rust/alloc/lib.rs
+++ b/rust/alloc/lib.rs
@@ -114,7 +114,6 @@
#![feature(const_eval_select)]
#![feature(const_pin)]
#![feature(const_waker)]
-#![feature(cstr_from_bytes_until_nul)]
#![feature(dispatch_from_dyn)]
#![feature(error_generic_member_access)]
#![feature(error_in_core)]
Speaking only for the toolchain side, no I would not like to wait for the kernel, nor maintain multiple toolchains. I think patching the kernel with newer support will be best, and I expect that will be easier if those changes already exist somewhere to cherry-pick -- e.g. if some linux branch tracks stable or even beta Rust.
Thanks! Yeah, the upgrades should be available as commits to cherry-pick, even in mainline, eventually (unless we are requested to provide the patches in a different way to upstream, but even then I will make it easy to see which ones are needed).
If the kernel does not track the latest Rust release, then we could still consider having upcoming upgrades somewhere in a different, rebasing branch, e.g. rust-rustc-upgrades
, which would reach mainline at some point in the future. No real difference from the kernel point of view, but at least the draft patches would be available early, like pre-merge when we tracked the latest Rust releases closely (sometimes I tested beta Rust too). It wouldn't be on linux-next, though.
Thanks @cuviper ... Built, tested and pushed https://gitlab.com/fedora-asahi/kernel-asahi/-/commit/8c5ce634fe9b66096146379b9e3b4b573adbf80d https://copr.fedorainfracloud.org/coprs/ecurtin/fedora-asahi-dev/build/5581873/
If people want to grab at some point here in upstream
Is there a general expectation of how often the Rust version used will be bumped?
Is there a general expectation of how often the Rust version used will be bumped?
Before we can declare a minimum version: ideally we would track the latest release, but it remains to be seen how kernel developers feel about it. On top of that, if klint support is merged and starts to be routinely used, then we will also need to be mindful of that.
After a minimum version is declared: if we follow a similar model to the GCC and LLVM support, which is likely, we will not track the latest release (i.e. as a minimum), though how wide the window will be remains to be seen. If users and distributions were happy with a small one, then it would be great for us to use the latest features, especially in the beginning. So we may start with a small window and then widen it, similar to what was originally decided for LLVM.
I think that as long as you use unstable features, you should be tracking the latest Rust compiler at their lifecycle. Once you're not using unstable features anymore, then widening the range of compilers supported is a reasonable conversation to have.
So this just happened again with Rust 1.70:
this will hurt distro adoption. Is there any update or a timeline for when R4L will keep up with the Rust release cycle?
I think from Fedora perspective, it's as important to keep up with the latest released maximum version than having a defined minimum version. Some sort of standardisation is needed.
So this just happened again with Rust 1.70:
This is expected. Given how unstable features are handled on the Rust project, it is always likely that the kernel side needs changes when you try newer Rust toolchains, until Rust stabilizes the features we need or until they give us stability guarantees in some other way.
If you want a branch with 1.70.0 support, then you may want to use https://github.com/ojeda/linux/tree/rust-1.70, though it is not finished -- there is at least a warning to be investigated.
this will hurt distro adoption. Is there any update or a timeline for when R4L will keep up with the Rust release cycle?
Please see https://lore.kernel.org/linux-patches/20230418214347.324156-4-ojeda@kernel.org/ for the latest news.
Several major rolling distributions are OK with us tracking closely the Rust releases. Others may decide to support a Rust package for kernel builds, like non-LTS Ubuntu did.
In any case, please note that the release schedules of both Rust and the kernel will not match exactly in any case, and that technical issues may always appear. This is why I started discussing with upstream Rust having tighter integration to, at least, detect them as early as possible, as a first step.
Moreover, please note that it may be a bit too early to start enabling Rust support for general-purpose distributions, since there are no in-tree users and some important features need to be disabled (that distributions likely want enabled) to make the Rust support work.
I think this issue can be closed now.
By the way, we have now a page for this: https://rust-for-linux.com/rust-version-policy
Description
Fedora 37/38 has moved to Rust 1.67.1, meaning you can't build Rust kernel modules on that platform anymore.
Architecture(s) any architecture
Toolchain versions
rustc
: rustc 1.67.1 (d5a82bbd2 2023-02-07) (Fedora 1.67.1-1.fc37)bindgen
: bindgen 0.63.0Kernel log
```text Paste it here -- feel free to reduce it to the relevant parts. ```
Kernel config
```text Paste it here -- feel free to reduce it to the relevant parts. ```