Closed withoutboats closed 2 years ago
I am able to create a struct:
#![feature(const_generics)]
struct MyArray<T: Default, const len: usize> {
real_array: [T; len]
}
impl<T: Default, const len: usize> MyArray<T, {len}> {
fn new() -> Self {
return MyArray {
real_array: [Default::default(); len]
}
}
}
This fails because I can't actually initialize the array due to:
error: array lengths can't depend on generic parameters
--> src/main.rs:9:46
|
9 | real_array: [Default::default(); len]
|
I've tried working around it by making const values set to the generic value.
That gets past the above error, but the compiler then crashes.
error: internal compiler error: src/librustc/ty/subst.rs:597: const parameter `height/#0` (Const { ty: usize, val: Param(height/#0) }/0) out of range when substituting substs=[]
The language desperately needs basic constant generics, such that the standard library doesn't have to define each function for each size of array. This is the only reason I don't use rust. Do we really need turing complete compile time functions? I might be wrong but simple integer expressions should suffice and I hope nobody is wasting their time, making sure that these wacky examples work.
The language desperately needs basic constant generics, such that the standard library doesn't have to define each function for each size of array
There is already some effort there https://github.com/rust-lang/rust/issues/61415.
I hope nobody is wasting their time, making sure that these wacky examples work.
Some people do, I cannot see any problem with that ;)
This is the only reason I don't use rust.
This is the least interesting reason I've seen quoted for someone not using rust. Are you sure you're telling the truth?
but simple integer expressions should suffice
Making even that reduced scope is incredibly hard. Try your hand at it, we have really capable people making this and there's a reason why it's taking this long.
Unfortunately, I've not been able to dedicate much time to fixing const generics bugs recently (or to Rust more generally). If anyone wants to get involved in pushing const generics along, I'd be happy to offer advice for tackling the open issues and review bug fixes, though it'll probably be a little while before I can concentrate on this again.
I hope nobody is wasting their time, making sure that these wacky examples work.
Nobody is, miri
is already far more powerful than C++ constexpr
.
And nobody is working on anything fancy like deriving N = M
from N + 1 = M + 1
.
Most of these bugs aren't about the expressions in them, they're about the typesystem and how const
generics interact with all other features.
They would still be there if all you had were const
generics and integer literals.
Btw, I think the "array lengths can't depend on generic parameters" error for [expr; N]
expressions aren't needed, we could use the same trick that we do for [T; N]
types, and pull out the N
without evaluating it as an expression.
While I do want to take a stab at this, I'm not sure I'm the right person. I've used little rust and know very little compiler theory. I might need a fair bit of coaching but I'm certainly willing. 😄
Edit: I have a fair bit of experience in software in general though.
@varkor, I've been looking for something useful to do on rustc
, and I'd love to step up and help out. Also, thanks for being candid about your ability to allocate time to it!
@varkor, I'm actually following through on the above comment, so if you have any advice, I'm all ears! Feel free to refer me to another communication channel, too.
One thing we could do right now, I just realized is allow Foo::<{...}>
/ [expr; ...]
expressions to refer to generic parameters (in the ...
part).
This is because expressions must be nested somewhere within a body, and that tends to prevent cyclic dependencies.
However, I'm worried that having e.g. [(); [0; 1][0]]
in a signature would break, so we'd probably need a crater run anyway (and nowadays those take forever).
For anyone who's interested in helping out with const generics, my advice would be to take a look at the list of open const generics issues and investigate whichever looks interesting to you. Some have had some investigation already, which should be in the comments. Most of the issues are probably going to require a little digging. It's helpful to make a comment if you're planning to investigate something, so we don't duplicate efforts (but you can often ignore the issue assignee if there hasn't been any activity for a while). If you have any questions, you can ask in the issue comments, or on Discord or Zulip. @eddyb, @yodaldevoid, @oli-obk and I are familiar with many of the relevant areas and are good people to ask. Thanks for all your interest!
cc @hameerabbasi, @ranweiler
Questions about const generics:
const
parameter of arbitrary type?const
generics are unimplemented / crash the compiler?P.S (Contributors:) Thank you so much for working on this feature.
const-generics is still under development so there aren't any official docs on it yet. I'm not too sure about the other two questions. When I last used const-generics they crashed when I specified some const-generic parameters, but it has been nearly a month since I did anything with it last so things may have changed since then.
And nobody is working on anything fancy like deriving
N = M
fromN + 1 = M + 1
.
This would be very useful to have a solver for such type constraints. For example when implementing an addition if two N-bits number, the return size should be (N + 1) to account for an overflow bit. When (for example) two 5-bit numbers are given the solver should check that N + 1 = 6
. Hopefully this can be bolted onto const generics later :)
@flip111 Yes, I think the plan is to add this later, but this sort of general expression constraints are very complex and hard to implement. So we may not see them for at least a few years.
To be honest, solving those kinds of constraint-based problems sounds a lot like a job for Prolog. Maybe piggybacking on the chalk engine is an option? Afaik it solves Traits, though, right?
Btw I love the topic, although I can't afford to help with this atm. Thank to everyone who's working on Const Generics for your time and commitment. 💐
A small update: I'm working my way up the Rust tree. I do plan to contribute but want to self-study to the place where I don't need excessive coaching.
I am able to create a struct:
#![feature(const_generics)] struct MyArray<T: Default, const len: usize> { real_array: [T; len] } impl<T: Default, const len: usize> MyArray<T, {len}> { fn new() -> Self { return MyArray { real_array: [Default::default(); len] } } }
This fails because I can't actually initialize the array due to:
error: array lengths can't depend on generic parameters --> src/main.rs:9:46 | 9 | real_array: [Default::default(); len] |
I ran into the same problem, but found a workaround using
MaybeUninit
: https://play.rust-lang.org/?version=nightly&mode=release&edition=2018&gist=3100d5f7a4efd844954a6fa5e8b8c526 It's obviously just a workaround to get properly initialized arrays, but for me this is sufficient until a proper way is made available. Note: I think the code should always work as intended but if someone finds a bug with the use of the unsafe, I would be happy to fix it.
@raidwas you are dropping uninitialized memory when you use the =
operator to initialize uninitialized memory. Do this instead,
@KrishnaSannasi thanks, good catch :D (technically I did not drop uninitialized memory since I only use it for primitives, but good to have a correct version here)
Technically, dropping even floats is undefined, but it isn't exploitable right now.
Technically, dropping even floats is undefined, but it isn't exploitable right now.
I would have expected that dropping any Copy
type, even uninitialized ones, is a no-op. Maybe that's worth changing.
Technically, dropping even floats is undefined, but it isn't exploitable right now.
I would have expected that dropping any
Copy
type, even uninitialized ones, is a no-op. Maybe that's worth changing.
It's still technically UB. While safely dropping defined Copy
values is a no-op, the compiler may decide to do some unexpected optimizations if you attempt to drop uninitialized memory of any type (e.g. remove all code that ever possibly touches that value), which could possibly break things. That's my understanding of it.
Not to be impolite, but ≥ 60 people get notified about comments on this thread, so we should probably keep the off-topic discussion to a bare minimum. Let's rather use this to coordinate the work on const generics as that's what we all want to happen.
Crashing if I used aliases pointing to type with const-param from another crate (dependency).
For example:
#![feature(const_generics)]
pub type Alias
- crate B
extern crate crate_a; use crate_a::Alias;
pub fn inner_fn(v: Alias
[crash-log (44580).txt](https://github.com/rust-lang/rust/files/3720635/crash-log.44580.txt)
@fzzr- sounds like #64730
Also sounds like https://github.com/rust-lang/rust/issues/61624
I've been switching some code to const generics and it seems like there are really two different use cases. I'm not sure if they should be conflated or if we'd be better served going with two different syntaxes for the use cases.
A lot of my usages are not actually for types where a constant value plays a role in determining the types, but rather to avail myself of features that are locked out for non-const/literal values (still not fully supported, e.g. match patterns, but ultimately will need to be).
IMHO we should formally land "const arguments" alongside const generics, so that people won't write mutant code introducing a thousand "const generics" (one for each argument) to get the compiler to evaluate certain variables as literals/constants.
@mqudsi Could you give an example? There are already plans and ongoing foundational work going on to make const eval more powerful. That's orthogonal to const generics, though.
What I mean is, if you want to refactor typical parser code like the following for reuse:
let mut result: u32 = 0;
let mut digits = 0;
while digits < max_digits && src.has_remaining() {
match src.get_u8() {
d@b'0'..=b'9' => {
result = result*10 + (d - b'0') as u32;
digits += 1;
},
b'>' => match result {
0..=191 => break, // valid range
_ => return Err(DecoderError::OutOfRange),
},
_ => return Err(DecoderError::Malformed)
}
}
You would typically move it out to a function, except the following won't work because certain values that are used in pattern matching have become variables:
#[inline(always)]
fn read_str_digits<B: bytes::buf::Buf>(src: &mut B, stop_word: u8,
max_digits: u8, min_value: u32, max_value: u32) -> Result<u32, DecoderError> {
let mut result: u32 = 0;
let mut digits = 0;
while digits < max_digits && src.has_remaining() {
match src.get_u8() {
d@b'0'..=b'9' => {
result = result*10 + (d - b'0') as u32;
digits += 1;
},
stop_word => match result {
min_value..=max_value => break, // valid range
_ => return Err(DecoderError::OutOfRange),
},
_ => return Err(DecoderError::Malformed)
}
}
...
}
If const generics lands before const arguments, I can suddenly abuse const generics to come up with the following:
#[inline(always)]
fn read_str_digits<const MinValue: u32, const MaxValue: u32, const StopWord: u8, B: bytes::buf::Buf>
(src: &mut B, max_digits: u8) -> Result<u32, DecoderError> {
let mut result: u32 = 0;
let mut digits = 0;
while digits < max_digits && src.has_remaining() {
match src.get_u8() {
d@b'0'..=b'9' => {
result = result*10 + (d - b'0') as u32;
digits += 1;
},
StopWord => match result {
MinValue..=MaxValue => break, // valid range
_ => return Err(DecoderError::OutOfRange),
},
_ => return Err(DecoderError::Malformed)
}
}
...
}
As of today, even this code won't work because the compiler does not detect the const generic values to be constants valid for use in a pattern, but that is obviously incorrect and needs to be addressed before RFC 2000 can land. Don't get me wrong, I've been fighting for generic constants for years and I have PRs ready to go for a dozen major crates (I can't wait to make the timezone in chrono
a const generic and unify all the various DateTime
types), but imho if it becomes possible to abuse const generics to fake const arguments (where by "const" what we really mean is "literal") then you're going to see widespread abuse of that.
It's not necessarily the end of the world, but without having dug too deeply into it, it does seem as if a proper and complete const generics implementation will necessarily include all the plumbing for const arguments anyway, so we might as well take the extra time to finalize the syntax/ux/story for const arguments while we're at it and avoid an unfortunate era of code stink. Yes, the above can still be done with macros, but the ergonomics of const generics are a thousand times easier.
fwiw, this is how I'd imagine the const argument version to look like:
#[inline(always)]
fn read_str_digits<B: Bytes>(src: &mut B, min_value: const u32,
max_value: const u32, stop_word: const u8, max_digits: u8) -> Result<u32, ()> {
let mut result: u32 = 0;
let mut digits = 0;
while digits < max_digits && src.has_remaining() {
match src.get_u8() {
d@b'0'..=b'9' => {
result = result*10 + (d - b'0') as u32;
digits += 1;
},
stop_word => match result {
min_value..=max_value => break, // valid range
_ => return Err(()),
},
_ => return Err(())
}
}
...
}
It would virtually be identical to const generics but for the semantics.
You would typically move it out to a function, except the following won't work because certain values that are used in pattern matching have become variables:
This seems like a use-case that is more readily addressed by allowing the values of bound variables to be used in patterns (not just consts), with some new syntax. The fact that consts are currently allowed in patterns is counter-intuitive and, as far as I'm aware, historical.
@mqudsi Forgive me if this is a silly question, but I don't see anything wrong about the example you gave. It looks like a perfectly valid use case for const generics to me: having a definition that is generalized to work with arbitrary values as the max/min. I don't really see the benefits of const arguments over const generics. They seem equivalent to me; that is, const arguments could just be implemented as a desugaring to const generics. Could you elaborate more on what's wrong with this design pattern?
Here's a summary of the work on const generics since the last update. Last time, it was all about the core implementation: making sure everything fitted together and getting some of the basic test cases passing. Since then, the effort has been focused on making const generics more reliable: fixing cases that crashed the compiler, or didn't work unexpectedly, or improving diagnostics. We're getting closer to something that works reliably, though there's still a way to go.
Additionally, other work on the compiler ended up fixing some of the other issues relating to const generics.
We've also started using const generics inside the compiler itself: array trait implementations now use const generics thanks to the work of @crlf0710 and @scottmcm (https://github.com/rust-lang/rust/pull/60466, https://github.com/rust-lang/rust/pull/62435), which led to performance improvements, and which will also allow us to unrestrict array trait implementations in the future (when const generics is stabilised). @Centril used the same approach to improve VecDeque
(https://github.com/rust-lang/rust/pull/63061).
Many of the particularly common bugs with const generics have been fixed in the past few months. @nikomatsakis is investigating lazy normalisation, which should fix a host of the remaining issues, while @jplatte is looking into fixing the disambiguation of const parameters from type parameters (so you'll no longer have to type {X}
for a const argument).
I also want to thank @eddyb for all their mentoring and reviewing for const generics throughout development, which has been invaluable.
There are still quite a number of other issues to address and, as before, we could use all the help we can get — if you're interested in tackling any of the remaining issues, feel free to ping me on the issue, or on Discord or Zulip for pointers on getting started.
@mqudsi @mark-i-m Would it be appropriate to extend the current syntactic alternative for generics, impl Trait
in argument position, to const generics? This is the syntax @mqudsi suggested for this use case.
Personally I think this would improve the readability of such usecases, but I see one problem: normal generics are used to pass data, so they are bound to an argument. This is not the case for const generics, so how would you pass them? Pretend they are arguments?
In the case of impl Trait
in argument position, it also cannot be used with the turbofish syntax. For const
in argument, it would be the inverse. This may be confusing.
I'm not sure if the benefits outweigh the "weirdness" here, but I wanted to share my thoughts anyway.
I vaguely recalled a discussion about this before, and now I have found it: https://internals.rust-lang.org/t/pre-rfc-const-function-arguments/6709
@PvdBerg1998 that seems like a very bad idea. I really don’t quite get the idea that <…>
is a hard syntax to read. Two possibilities here:
where
clause?Now compare with the pre-RFC linked above. It would be introducing a tag (in the sense of the type-system) directly into the position of function arguments, i.e. variable bindings for the body of a function.
In other terms: it’s very confusing and, as several people stated for impl Trait in arg position, it’s not needed. Please don’t make the same mistake twice to add a feature that is not worth it. Especially if you “pretend they’re arguments.” Rust would be going in the wrong direction here. The syntax should be ligther, yes, but not more confusing. Please stop.
@phaazon
I personally fully agree with the linked proposal that allowing constants in argument position will be a really good ergonomic boost. It's not only about function signatures (arguably where
clauses do not "solve" the problem at all, but will make it harder to understand signatures, since you have to parse 3 places instead of just one), but also about how those functions will be used. Compare:
let r = _mm_blend_ps::<{2 * C}>(a, b);
let r = _mm_blend_ps(a, b, 2 * C);
The second line is much more natural in my opinion than the second one. Another example would be matrix creation: MatrixF32::new(3, 3)
vs. MatrixF32::new::<3, 3>()
. I strongly disagree that this functionality will be "very confusing". For user it's nothing more than an additional requirement for a function argument. We already can emulate const arguments via macros (see SIMD intrinsics inside std
), but it's quite ugly and inefficient.
Regarding impl Trait
in argument position, initially I was also against this feature, but after some time I've changed my mind and I believe it was a good decision in the end. The only incomplete part right now is interaction with explicit type parameters provide via turbofish. I think a good solution would be to make impl Trait
arguments invisible for turbofish, i.e. users should use it when they are sure that explicit type disambiguation will not be needed. It would allow us to significantly reduce a need for _
inside turbofish in some scenarios.
I guess a discussion of const arguments is off topic here, so probably should not continue it here.
One thing that separates const
arguments from impl Trait
arguments (which @PvdBerg1998 mentioned) is that they are arguments, with all that that implies for type inference.
You can't pass a type itself as an argument, but you can pass a value, and letting a function infer its const generic arguments from its normal parenthesized arguments is completely reasonable.
This is fairly common in C++, for example:
template <typename T, size_t N>
size_t length(T (&array)[N]) {
return N;
}
This is subtly different from simply accepting a const parameter and desugaring it to a const generic parameter (fn foo(const N: usize)
-> fn foo<const N: usize>()
). It's closer to constraining a parameter with a const generic parameter, and then inferring that const generic argument from the argument. So the matrix example above might look like this:
impl<const W: usize, const H: usize> MatrixF32<W, H> {
fn new(w: usize<W>, h: usize<H>) -> Self { .. }
}
...where usize<X>
is made-up syntax for "a usize
with the value X
." (Analogously, you might want usize<X..Y>
for ranged types, which have been discussed in the context of generalizing NonZero
.)
If you wanted to avoid roping in ranged types, you might look at dependent-typed languages which allow parameters to be used directly in types, with a bit of scope tweaking:
fn new(const W: usize, const H: usize) -> MatrixF32<W, H> { .. }
They are not arguments. They are generics. On a type system level, that means their values live at compile-time, while a function argument’s values live at runtime. Mixing both is extremely misleading.
As I care about type systems, this proposal goes against a lot of sane and sound principles. You don’t want to mix values and types. Because, yes, a N
(note I didn’t say usize
, I said N
), even though you think it’s a value, is way more akin to a type than a value. As a rationale, Haskell doesn’t have const generics but it has DataKinds
, allowing to lift regular data constructors as types:
foo :: Integer -> Integer
foo 0 -- 0 has type Integer
-- but
data P (a :: Integer)
type MyP = P 10 -- 10 has kind Integer, which “value” is the 10 type
So:
fn foo<const X: usize>()
Here, X
is more akin to a type than a value.
Also, I care about monomorphization. If I read this:
let x = foo(12, "Hello, world!", None);
With your proposal, if we look up at the definition of foo
, it’s hard to know which arguments are going to monorphize foo
. Because everytime you pass a different value for a const generic, you create a complete new function.
Maybe it feels more intuitive for you and your reasons, but I also have reasons to say it’s non-intuitive at all in terms of type correctness. Stating that they are arguments is like stating that parametered functions, in math, have their parameters arguments. You are confusing parameters and arguments, demonstrating how bad that idea is.
You may have misread me, I specifically said "const
arguments ... are arguments," not "const
generics." My first Rust example is also exactly equivalent to DataKinds (usize<N>
is the type-level version of N
). There is no problem here in terms of type correctness- I'm not arguing in terms of intuition but by analogy to existing, established, and sound type systems.
As for your concern about monomorphization- that's no different than today! Every argument can potentially cause a new monomorphization. I would also add that the solution to this problem is to reduce the cost of all generics by sharing non-dependent code between instances, including-and-especially by converting const generics to runtime values when possible.
(And I'm utterly mystified by your claim that I'm confusing parameters and arguments, I was careful to distinguish them.)
I get what you mean but I still feel very concerned. Because the argument is tagged const, it means you cannot pass it a value, runtime speaking. You need to pass it a constant value, which will map to the const generic. By the way, you didn’t mention it here but that’s just an application of a singleton type if you consider 10
as a type: its single possible value is 10
.
As I’ve been against impl Trait in arg position, I’m against that one too (and you’ll get why we can compare here). Several points:
<…>
is harder than (…)
, especially considering struct
vs fn
. Currently, struct
has only <…>
and fn
has both, because they both can be set parameters on.foo<A, B>
, you know there are two parameters that will yield to a function definition and implementation.I feel what you want to do here. foo(1, 3)
feels better for you than foo<3>(1)
, but not for me. The reason for this is that foo(1, 3)
should accept the second argument to be runtime-based and the proposal you give forbids it. I can see applications, though, especially with arrays, but I currently don’t like what it looks like. I would be much more for a solution that says “if an argument is const
, we can infer a type variable / const generic.
fn foo<const N: usize>(a: usize, b: usize | N);
That’s just imaginary syntax I came up with to explain my point: if you pass b
as a const value, N = b
and you end up with your nice syntax:
foo(1, 3)
If b
is not a const value, it’s required that you pass N
explicitly:
foo<3>(1, x)
Also, what would be the syntax for this, with your proposal:
fn foo<const N: usize>(x: [f32; N]);
fn new(const W: usize, const H: usize) -> MatrixF32<W, H> { .. }
It leads us to question about associated items, in short
new()
is asociated function, and must be asociated with concrete type, const generic items have semantic like "This type having this param...", i.e. dependent type. So,new()
in the context shouldn't have any arguments. Explict usage should look likeType<32>::new()
. And of course all const generics arguments should be elidable.
I feel what you want to do here. foo(1, 3) feels better for you than foo<3>(1), but not for me. The reason for this is that foo(1, 3) should accept the second argument to be runtime-based and the proposal you give forbids it.
That is not my motivation nor intention- I'm not talking about what "feels better" and I don't believe the compile/run-time distinction needs to be conflated with the type/value-level distinction. The two are already separate- const
items (value or fn) are certainly not types!
All I'm getting at is the semantic connection that type inference draws between value-level parameters and type-level parameters. There is no relation to impl Trait
- my point, again, was to distinguish const arguments from impl Trait
, precisely to head off this sort of unproductive angst.
Also, what would be the syntax for this, with your proposal:
fn foo<const N: usize>(x: [f32; N]);
That syntax wouldn't change! I'm not proposing we remove const parameters from the type level (syntactically or semantically), just that we add them to the value level, partially to aid inference of the type-level ones.
At cppcon there was a talk about constexpr function parameters, which seems similar to what @PvdBerg1998 is talking about and seems just like the internals thread that @mark-i-m linked to. It seems to be well received there.
It seems like a good idea, but it should be worked out after we get the initial implementation for const-generics done
CppCon 2019: David Stone - Removing Metaprogramming From C++, Part 1 of N: constexpr Function Params
Edit: This was mentioned in that thread, here is the related paper for constexpr function parameters https://github.com/davidstone/isocpp/blob/master/constexpr-parameters.md
I would like to request that we reserve the discussion of such a significant syntactic change for either a thread on internals or an issue (or amending PR) on the RFCs repo. This issue's comment section is already quite long and the syntax in question seems unnecessary for a usable MVP of const generics.
If we could move comments we probably would. We might want to lock the issue to @rust-lang team members and hide all the offtopic comments?
All design discussion should happen on the RFC repo and all bug reports should be in separate issues.
cc @rust-lang/moderation Is there precedent for doing this?
You can convert a comment to an issue, but not move comments. Locking is fine, and i'd just hide off topic comments.
With the start of the new year, I thought it would be good to give another update on where we are now with const generics. It's been a little over 2 years since the const generics RFC was accepted. In the first year (roughly 2018), a series of large refactoring efforts were undertaken to facilitate const generics by improving the handling of generic parameters generally throughout the compiler. In the second year (roughly 2019), work began on implementing const generics itself: first by getting the bare minimum working, and then by slowing improving the integrity of the feature. People also started to experiment with using const generics in the compiler itself. More detailed descriptions of these efforts are in the first two update posts: [1], [2].
There's been good progress in the last couple of months since the last update.
{}
(https://github.com/rust-lang/rust/pull/66104)structural_match
checking to forbid types with custom equality checking from being used as const generics (https://github.com/rust-lang/rust/pull/65627)A big thanks to everyone who has been helping out with const generics!
What's next? The biggest blocker for const generics at the moment is lazy normalisation, which is required for certain kinds of const generic bounds (such as in arrays). @skinny121 is currently investigating lazy normalisation, continuing their fantastic efforts taking down the big bugs in const generics. This would address many of the present issues with const generics.
Lazy normalisation aside, there are still a sizeable number of bugs and papercuts we'd like to address. If you're interested in tackling any of the remaining issues, feel free to ping me on the issue, or on Discord or Zulip for pointers on getting started. I'm confident we can make good headway in 2020 and hopefully approach a point where stabilisation becomes a viable discussion!
Tracking issue for rust-lang/rfcs#2000
Updates:
If you want to help out, take a look at the open const generics issues and feel free to ping @varkor, @eddyb, @yodaldevoid, @oli-obk or @lcnr for help in getting started!
Blocking stabilization:
Remaining implementation issues:
FIXME(const_generics)
comments.FIXME(const_generics_defaults)
).has_infer_types
.{X * 2}
.