rust-lang / rust

Empowering everyone to build reliable and efficient software.
https://www.rust-lang.org
Other
98.69k stars 12.75k forks source link

Tracking issue for a minimal subset of RFC 911, const fn #53555

Closed Centril closed 6 years ago

Centril commented 6 years ago

This is a tracking issue for the RFC "Const functions and inherent methods" (rust-lang/rfcs#911).

This issue only tracks a minimal subset of the proposal in 911 that we are (hopefully) comfortable with stabilizing. To opt into the minimal subset, use #![feature(min_const_fn)]. To use the more expansive feature set, you can continue using #![feature(const_fn)] and other associated feature gates.

The minimal set will not include items from the following (incomplete) list:

  1. const fns with type parameters with bounds (including where clauses) in scope (including from the parent e.g. impl) other than: lifetimes, Sized, or (the "un"-bound) ?Sized

    This restriction exists because we are not sure about our story around what bounds mean in a const fn context. See https://github.com/rust-lang/rfcs/pull/2237, https://github.com/rust-rfcs/const-eval/issues/1, and https://github.com/Centril/rfc-effects/ for a discussion on this.

  2. const fns with argument types or return types that contain fn pointers, dyn Trait, or impl Trait.

    This is checked recursively. The restriction ensures that you may not reach a value of these types by any means.

    This restriction exists for the same reasons as in 1.

  3. const fns with any operations on floating-point numbers. This is achieved by making any floating-point operation not be const inside const fn.

    This restriction exists because we are not sure about the story wrt. determinism, achieving the same results on compile-time / run-time (including other machines) and floating points.

  4. using a const fn call in a pattern, e.g.;

    const fn twice<T: Copy>(x: T) -> (T, T) { (x, x) }
    match x {
        twice(21) => { ... }
        _ => { ... }
    }
  5. anything else that is not currently in const_fn or constants

    • raw ptr to usize cast (e.g. *const/mut T -> usize).
    • raw ptr deref.
    • if / if let / match.
    • loop / while.
    • let and destructuring.
  6. union field access.

  7. code requiring unsafe blocks.

Exhaustive list of features supported in const fn with #![feature(min_const_fn)]:

  1. type parameters where the parameters have any of the following as part of their bounds (either on where or directly on the parameters):

    1. lifetimes
    2. Sized

    This means that <T: 'a + ?Sized> and <T: 'b + Sized> + <T> are all permitted. Note that ?Sized is the absence of a constraint when bounds have been fully elaborated which includes adding implicit Sized bounds. This entails that permitting Sized + lifetimes allows the above examples.

    This rule also applies to type parameters of items that contain const fns.

  2. arithmetic operators on integers

  3. boolean operators (except for && and || which are banned since they are short-circuiting).

  4. any kind of aggregate constructor (array, struct, enum, tuple, ...)

  5. calls to other const fns (methods and functions)

  6. index operations on arrays and slices

  7. field accesses on structs and tuples

  8. reading from constants (but not statics, not even taking a reference to a static)

  9. & and * (only dereferencing of references, not raw pointers)

  10. casts except for raw pointer to usize casts

  11. const unsafe fn is allowed, but the body must consist of safe operations only

The bar for stabilizing const fns in libcore/liballoc/libstd will be that they are writable in stable user code (unless they are wrappers for intrinsics, i.e. size_of and align_of). This means that they must work with min_const_fn.

Things to be done before stabilizing:

Unresolved questions:

None.

Vocabulary:

Centril commented 6 years ago

Also cc: @eddyb, @RalfJung, @est31, @durka, @mark-i-m, @japaric

RalfJung commented 6 years ago

const fns with type parameters with bounds

This includes where clauses I assume?

integration with patterns.

I do not understand what this means. What will not work?

7.

Was anything supposed to go there?

Centril commented 6 years ago

@RalfJung

This includes where clauses I assume?

Indeed. Also arg: impl Trait and -> impl Trait should probably also be verboten.

I do not understand what this means. What will not work?

From the #24111 issue:

Integration with patterns (e.g. https://gist.github.com/d0ff1de8b6fc15ef1bb6)

Was anything supposed to go there?

I assume so 😆 I'm waiting for @oli-obk to complete their fleshing out of my initial issue description :)

Centril commented 6 years ago

Some concerns noted over at #24111:

oli-obk commented 6 years ago

runtime-pointer-addresses (https://github.com/rust-lang/rust/issues/24111#issuecomment-386745312)

does not apply here, because we neither stabilize union field accesses which could be used to transmute between any pointer and an integer, nor do we stabilize casting *const T to usize.

priority (https://github.com/rust-lang/rust/issues/24111#issuecomment-376652507)

The 2018 edition takes precedence over this feature. Since it has been minimized to a very small surface area which will be enforced by a feature whitelist, it seems plausible that we can get everyone on board to at least stabilize it shortly after the 2018 edition has been released.

design (https://github.com/rust-lang/rust/issues/24111#issuecomment-376829588) everything-const (https://github.com/rust-lang/rust/issues/24111#issuecomment-376652507)

Since we are punting on (almost) all design issues by using a minimal scheme, the only design question left is whether we want to start attaching the const keyword to many functions.

Alternate designs:

Wrt the "everything function is getting const attached to it" worry: The current design is entirely forward compatible with allowing const mod, const impl to allow marking entire code regions as const. Although that seems like it could benefit from the attribute, as you could write #![const] at the top of the crate root and have everything be const.

RalfJung commented 6 years ago

because we neither stabilize union field accesses which could be used to transmute between any pointer and an integer

We do not? I see you sneakily added that to the first post.^^

These are already stable for const. Why not allow them in const fn?

Centril commented 6 years ago

@RalfJung

Why not allow the in const fn?

@oli-obk added that part, but I think about it like this: given union field accesses, you can perform &[mut] T -> usize and so you may violate our plans for CTFE correctness per your blog post. Now, we can say that the operation *(const/mut) T -> usize is UB in const fn, but I would like to wait with that because:

  1. It gives us more time to develop our story around the rules of unsafe within const fn.
  2. We need to make sure that our teaching material is up to the task so that users will not do &T -> usize and think that's OK.

All in all, I think that if we are more conservative here and punt on this, we can ship a more minimal const fn faster.

RalfJung commented 6 years ago

I see. I don't like these inconsistencies but oh well. Looks like qualify_consts will be cleaned up... another time.^^

In terms of teaching, again we have the same problem for const... I thought the sanity check might help (unlike a const fn, a const has a very simple "observable behavior" so we can actually test for these things statically) but it seems it does not complain about

union Trans { x: &'static i32, y: usize }
const BAR : usize = unsafe { Trans { x: &0 }.y };
nikomatsakis commented 6 years ago

First, a nit:

const fns with type parameters with bounds (including where clauses) (lifetimes are OK) either on themselves or on the scope they are contained within (e.g. impls and such...)

Presumably something like T: Sized is ok -- or, more to the point, const fn foo<T>(t: T) -> T { t }...? So is it just that Sized trait is allowed?

But, more generally, I think that this proposal does a good job of outlining the surface area of the language to be stabilized, but I'd like to see a better outline of what the implications are regarding the "const system in general".

One key question here is the interaction of const fn and promotion, I think, since that is the main place where we can find ourselves in tricky questions (e.g., even something like const fn foo(a: u32, b: u32) -> { a + b } can be problematic if a call to it is promoted). Can someone (e.g., @oli-obk) write out the conditions in which calls to const fn can currently be promoted, if any? I don't recall. =)

I think by now I am mostly bought into the general framing from Ralf's blog post, but I'd like to also know if there are alternative approaches that we are ruling out by stabilizing const fn.

It seems like one of the key bits that I could imagine being controversial was the idea that a const fn could do things that we cannot statically guarantee are valid (e.g., in unsafe code) but that we could still promote calls to that const fn. If we wind up hitting errors at compilation time, that is then a violation of the const fn declaration analogous to hitting UB at runtime. As I said, I think I am mostly bought into this framing, but I would like to clarify if we are losing room to maneuever here in any way.

nikomatsakis commented 6 years ago

To expand a bit on something:

Presumably something like T: Sized is ok -- or, more to the point, const fn foo(t: T) -> T { t }...? So is it just that Sized trait is allowed?

I had initially considered suggesting that we allow auto traits as well, but I do not think this is a good idea. For one thing, unlike Sized ,auto traits may have custom impls, and that might constrain us in ways that we do not want. e.g., you can have

unsafe impl<T: Display> Send for Foo<T> { }

Now, if you have a const fn foo<U: Send>(...) where U = Foo<T>, and we accept it, we will therefore be saying that T: Display is provable in a const context. But if we wanted to change how trait resolution works in a const context to consider only impls that are marked const or some such thing, that could be problematic.

Since Sized does not permit custom impls, we don't have this problem. Similarly, we could permit lifetime bounds (T: 'a).

Centril commented 6 years ago

@RalfJung

I see. I don't like these inconsistencies but oh well.

Hehe; I don't like them either, but I fully intend for many of the restrictions to be temporary ^.^


@nikomatsakis

So is it just that Sized trait is allowed?

Right; so I think lifetimes in bounds + Sized + ?Sized (which is the true "no bound") would be permitted.

mark-i-m commented 6 years ago

I'm curious about the UX side of things. Some of the limitations might be surprising to people. For example, being able to write a generic but not bounds.

I can imagine it being frustrating to start implementing my program and then all of a sudden something doesn't work because it requires nightly. I either have to change to nightly or change my implementation approach.

Of course, it could be one of those things that just needs experimentation on nightly for a while. For example, we could implement the proposed min_const_fn feature and see how many uses of the const_fn feature on crates.io can be changed to it. If that proportion is low, I think we should think twice before stabilizing.

RalfJung commented 6 years ago

Can someone (e.g., @oli-obk) write out the conditions in which calls to const fn can currently be promoted, if any?

My long-term plan (which I still did not get around to submit to the const RFC repo...) was that a call to a safe const fn would get promoted whenever all its arguments would get promoted. Notably, we do not look into the body.

I originally thought we could not look into the body for promoting uses of const either, but IIRC there are problems then because we do want to promote

const FOO : Option<Cell<i32>> = None;
&FOO

but not

const FOO : Option<Cell<i32>> = Some(Cell::new(0));
&FOO

So it seems like, even if it's just for backwards compatibility, looking "into" a const body is needed. I hope that what we are really doing is just look at the result of the const, i.e. the final value, not the expression used to compute it (and TBH I do not know how we check if that value is Sync, but whatever). But I think we should not do that for const fn (and, with their argument dependencies, that would be indeed much harder).

Now @oli-obk can fix all my misconceptions. :D

oli-obk commented 6 years ago

Can someone (e.g., @oli-obk) write out the conditions in which calls to const fn can currently be promoted, if any?

A const fn call is "always" promoted right now. If the creation of its arguments involves unions, raw pointer to usize casts or raw pointer derefs, then the analysis doesn't even consider the the call for promotion. So it's not a question of whether to promote a const fn call, but whether its arguments seem promotable.

min_const_fn does not change this, but it also restricts the function body to a minimal setting where you cannot write a const fn that would cause it to fail promotion. Any potential unconst feature has been removed.

This means we are fully forward compatible with any scheme that we choose wrt promotion failures (or lack thereof).

So it seems like, even if it's just for backwards compatibility, looking "into" a const body is needed.

We don't allow even mentioning cells, even if they don't end up in the final result. I think it would be very confusing for users if they can use Cell in a const fn body to compute something, and return a u32 and then that result is not promoted.

    const FOO : Option<Cell<i32>> = (None, Some(Cell::new(0))).0;
    let _: &'static _ = &FOO; // ERROR: does not live long enough

If we want to make this properly (meaning not surprising to users) we need to do these checks on final values, not on constant/const fn bodies. I have implemented a prototype for checking whether a value needs user-written Drop. I think I can also do this for UnsafeCell.

I'm curious about the UX side of things. Some of the limitations might be surprising to people. For example, being able to write a generic but not bounds.

We have so many const features that users are surprised about not being stable although very similar features are. I don't think extending the group of stable features will make ppl unhappy.

RalfJung commented 6 years ago

We don't allow even mentioning cells, even if they don't end up in the final result.

So we check the type to be frozen recursively?

I have implemented a prototype for checking whether a value needs user-written Drop. I think I can also do this for UnsafeCell.

Oh so currently we do not do any value-based analysis, and could go entirely off of the type of the const, not entering its body?

No that's not true, the following works

    const FOO : Option<Vec<i32>> = None;
    let _: &'static _ = &FOO;
oli-obk commented 6 years ago

could go entirely off of the type of the const, not entering its body?

No, I implemented a prototype for checking the constant's final value for promotability. So the body is ignored, but we check the value similarly how the const sanity checks work.

RalfJung commented 6 years ago

Well that's still looking "into" the constant, breaking the abstraction.

OTOH it enables reasonable code, so we likely want it. You said you "implemented a prototype", but something seems to already have landed?

oli-obk commented 6 years ago

You said you "implemented a prototype", but something seems to already have landed?

No, we've had body-revealing checks since forever. This is nothing new.

durka commented 6 years ago

Why can we not allow control flow in const fns?

durka commented 6 years ago

Also, has there been an audit of which const fns in std/crates.io can be implemented according to these restrictions?

Centril commented 6 years ago

@durka I've skimmed the const fns in libstd and I think it should all work with these restrictions. However; if it does not work for some of the stable ones, we will have to create some sort of exemption mechanism for those (think rustc_const_stable). A more thorough audit of libstd would be good.

Wrt. control flow and destructuring, see:

oli-obk commented 6 years ago

@durka this is not the issue for discussing what to add to the minimal const fn (control flow isn't even in full const fns yet), but for discussing what to remove to make it forward compatible to every possible scheme.

So. @RalfJung has noted that NonZero::new_unchecked is not const safe, so the question is whether to ban unsafe const fn under this feature gate.

RalfJung commented 6 years ago

Instead of banning random subsets of unsafe operations, I'd prefer -- if we want to keep the space open for how we resolve "const safety" -- to just not allow unsafe blocks in const fn, period. That would cover both union field access, raw pointer deref and calling unsafe const fn.

I think the only safe operation we are worried about is raw ptr to usize, and AFAIK that is not even allowed in const, so we are good.

Centril commented 6 years ago

Update:

Pratyush commented 6 years ago

If I understand correctly, one still cannot mark trait functions as const. If so, how is integer arithmetic const? Doesn't the implementation go through the arithmetic traits?

Centril commented 6 years ago

@Pratyush so what happens essentially is that we route integer arithmetic and other overloaded operations on primitive types to their intrinsic operations. For example, see the comment here.

You can observe that integer arithmetic works fine in const contexts because

const _FOO: usize = 42 + 1337;

is legal on a stable compiler. This works despite Add::add not being defined as const fn.

Centril commented 6 years ago

@rfcbot merge

I believe we have carved out a quite conservative and sufficiently minimal subset of const fn in this #![feature(min_const_fn)] (per description in the issue OP) that we can ensure that all open design questions will remain open in the future and that we do not foreclose on possibilities we want to preserve for ourselves. With respect to the bits that we do decide on here and the options that are settled, a rationale is given by @oli-obk in https://github.com/rust-lang/rust/issues/53555#issuecomment-414647732.

@nikomatsakis noted that:

I'd like to see a better outline of what the implications are regarding the "const system in general".

This was then done by @oli-obk in https://github.com/rust-lang/rust/issues/53555#issuecomment-414698907 where they noted in particular that:

This means we are fully forward compatible with any scheme that we choose wrt promotion failures (or lack thereof).

@mark-i-m noted wrt. UX that:

Some of the limitations might be surprising to people. For example, being able to write a generic but not bounds.

I personally think that this is bound to happen. However, @oli-obk noted that:

We have so many const features that users are surprised about not being stable although very similar features are. I don't think extending the group of stable features will make ppl unhappy.

I very much agree with this sentiment. Particularly, I think that we must start somewhere because otherwise, we will likely get nowhere (RFC 911 was merged 3+ years ago, that's long enough...).

I think the natural next step after stabilizing min_const_fn would be to focus on getting control flow (if, match, loop, while, ..) working. That I believe would open up a lot more abilities for users, and it also does not directly touch upon issues with bounds which have been thorny.

Finally, @durka noted that:

Also, has there been an audit of which const fns in std/crates.io can be implemented according to these restrictions?

I believe that at least for the standard library, this should be feasible to do in short order (I'll get right on that).


If we are fast enough to implement the feature gate, the tests, and so on, I believe it might be possible to ship this with the Rust 2018 release. That would be quite the achievement and shine a positive light on Rust I believe.

However, while the implementation and restriction tests are pending, let's note:

@rfcbot concern implementation

and

@rfcbot concern tests

rfcbot commented 6 years ago

Team member @Centril has proposed to merge this. The next step is review by the rest of the tagged teams:

Concerns:

Once a majority of reviewers approve (and none object), this will enter its final comment period. If you spot a major issue that hasn't been raised at any point in this process, please speak up!

See this document for info about what commands tagged team members can give me.

TimDiekmann commented 6 years ago
Exhaustive list of features supported in const fn with #![feature(min_const_fn)]:

...

  1. integer arithmetic.

Does this include wrapping_add, checked_add, rotate_left/right etc.?

oli-obk commented 6 years ago

Does this include wrapping_add, checked_add, rotate_left/right etc.?

No, this only includes basic integer operators for now

TimDiekmann commented 6 years ago

No, this only includes basic integer operators for now

Thanks! Will they be added at a later point then?

oli-obk commented 6 years ago

This issue is about the very minimal const fn feature. Any additional extensions should be discussed in their own issues or on the const fn tracking issue (https://github.com/rust-lang/rust/issues/24111)

But yes, there's basically no future where we'll not be making them const fn

RalfJung commented 6 years ago

boolean operators (&& and || are treated as & and | respectively).

Please don't edit the issue text without saying you are doing this, otherwise nobody will notice this.^^

How do you plan to do this, and isn't this incorrect because it does not short-circuit? I'd rather not support them than supporting them in this strange way.

It'll hopefully not be too long until we allow if, then we'll also get && and || (by allowing SwitchInt). And vice versa it seems to odd to have && and || and not if.

oli-obk commented 6 years ago

Please don't edit the issue text without saying you are doing this, otherwise nobody will notice this.^^

I feel very innocent here. I assumed it was an oversight since I told @Centril about this before the rfcbot was triggered while we were still wildly editing the issue text.

How do you plan to do this, and isn't this incorrect because it does not short-circuit? I'd rather not support them than supporting them in this strange way.

I'll see how to disable them entirely. Note that constants already fully support them without having short circuiting behaviour. See https://play.rust-lang.org/?gist=48a7eaa253d1ff0467b4e4b4f1c564d4&version=nightly&mode=debug&edition=2015 for the current behaviour (even at runtime)

#![feature(const_fn)]
#![allow(const_err)]

const fn foo() -> bool { false && [true][10] }

fn main() {
    foo(); // panics due to index out of bounds
}
Centril commented 6 years ago

@oli-obk

I feel very innocent here.

Alright; I'll happily take the blame... ;)

So we'll ban && and || in min_const_fn so that we don't introduce broken behavior in const to const fn.

@eddyb noted that we can do this by expanding && and || in HAIR to if foo() { bar() } else { false } which is already banned in min_const_fn and gated differently; so problem solved.

I'm editing the issue description to reflect this.

est31 commented 6 years ago

@oli-obk is there a way where this non-short-circuiting behaviour is observable without there being an error? I mean: the const system right now is very much void of mutability and side effects. If the only way the non-short-circuiting behaviour is observable is through errors, then a change to make && short-circuit wouldn't be a breaking one. At least as long as this change is performed before any mutability is being allowed in the language.

oli-obk commented 6 years ago

@est31 while it isn't a breaking change, it is very surprising, especially since const fn, in contrast to statics and constants, may happen to never be evaluated at compile-time. The solution chosen now will simply force users to use & and |, because && and || just won't compile. The change is trivial and feels correct to me.

est31 commented 6 years ago

@oli-obk you misunderstood my suggestion: I didn't suggest that const fn should get && and then it should become short-circuited. I suggested that we make the && short-circuited in statics and constants before it is too late and they get the ability to mutate. This is unrelated to const fn per se (yes we are in the issue thread of const fn but it's here where the && behaviour came up). Is there an issue about this change, so that it is not being forgotten about?

oli-obk commented 6 years ago

So... wrt the part about forbidding unsafe code:

I suggest we do allow unsafe functions that do no unsafe operations whatsoever. This would allow us to make NonZero*::new_unchecked stable without any hacks and allow users to write their own unsafe functions that are only unsafe to call because of what they provide, not because of what they do.

I have implemented a check for this by making the unsafety check (that rustc already does) emit a special unsafety message not talking about "needing an unsafe block" but about "unsafe ops not being allowed in const fn". This check completely bypasses the "are we inside unsafe code" checks and is unconditionally executed for min_const_fn functions.

oli-obk commented 6 years ago

I suggested that we make the && short-circuited in statics and constants before it is too late and they get the ability to mutate

We'll just block stabilizing let bindings on making && and || short circuit I guess

oli-obk commented 6 years ago

I adjusted the implementation to allow const unsafe fn, but only if the body of the function is entirely safe. This means these unsafe functions cannot even call other const unsafe fn, even though the very same analysis basically guarantees that this should be fine.

RalfJung commented 6 years ago

This would allow us to make NonZero*::new_unchecked stable without any hacks

That works because it's "just" a struct constructor that is special because of a lang item? I'd say this is a hack, but it seems reasonable.

However, I'd like to see a test ensuring that

const FOO: NonZeroU8 = unsafe { NonZeroU8::new_unchecked(0) };

is caught by the sanity check. Currently nightly accepts that constant without complaining.

Centril commented 6 years ago

@rfcbot resolve implementation @rfcbot resolve tests

Implementation and tests were added in #53604.

rfcbot commented 6 years ago

:bell: This is now entering its final comment period, as per the review above. :bell:

mark-i-m commented 6 years ago

So since it looks like this will be accepted, would the next step be to try to systematically replace uses of the const_fn feature with this feature? That would allow us to get a preliminary survey of what common user cases are.

oli-obk commented 6 years ago

@mark-i-m we could start linting crates which use the const_fn feature gate but min_const_fn would have been sufficient

I want to block stabilization on https://github.com/rust-lang/rust/pull/53851 (don't promote all const fns, otherwise making a function const fn can break runtime code)

SimonSapin commented 6 years ago

I’d be glad to see incremental progress with any subset of const fn being stabilized, but the min_const_fn subset as currently implemented in Nightly seems overly conservative. (More than the initializers of const and static items in Stable.) I hope we can extend it quickly.

I’ve tried to replace #[feature(const_fn)] with #[feature(min_const_fn)] in Servo and ran into some limitations:

  1. Trait bounds are not allowed even if they’re not relied on in the function’s body:
pub struct Guard<T: Clone + Copy> {
    condition: Condition,
    value: T,
}

impl<T: Clone + Copy> Guard<T> {
    pub const fn new(condition: Condition, value: T) -> Self {
        Guard {
            condition: condition,
            value: value,
        }
    }
  1. Function pointer types are not allowed even in enum variants where only other variants are used, for example to initialize an Option<fn(Foo)> field to None.

  2. Values of function pointer types are not allowed even when they are not called, for example to initialize an fn(Foo) field based on an fn bar(foo: Foo) {…} item.

All of the const fns in question are trivial constructors for structs with private fields. They exist to allow static items of those types to be created in different modules. However if the statics were moved into a module where the fields are visible, and the constructors were inlined, this code would be allowed today without any feature gate.

oli-obk commented 6 years ago
  1. Can be solved by removing the bounds from the type and adding it to impls only. Then you can add an impl<T> Guard<T> just for the constructor function
  2. and 3. This is an intended restriction. I personally don't think it's useful, but there's an argument for allowing to call const fns via function pointers to them. The argument is that we might not want an explicit const fn pointer that only requires const fn if called at compile-time, but at runtime takes normal functions just fine.
Centril commented 6 years ago

@SimonSapin

I hope we can extend it quickly.

I also hope we will; in particular with control flow and let bindings.

However, as for trait bounds, auto traits, and function pointers it is currently highly unclear what the design should be.

In particular, does const fn foo<T: Clone>(...) { .. } mean that T has a "const implementation" of Clone or is any current clone implementation permissible?

Also, does const fn foo(x: fn() -> u8) -> u8 { .. } mean that any fn() -> u8 will do, or must it be a const fn() -> u8.

Finally, what does const fn foo(x: Box<dyn Bar>) -> u8 { .. } mean? Will it accept any trait object or must it be a const implementation of Bar?

For these parts, don't expect to stabilize anything quickly.

mark-i-m commented 6 years ago

If a const fn is never used at runtime, does it even end up in the binary? How does this work with function pointers exactly?