search

rust-lang / rust

Empowering everyone to build reliable and efficient software.
https://www.rust-lang.org
Other
98.38k stars 12.72k forks source link

const fn tracking issue (RFC 911) #24111

Closed nikomatsakis closed 5 years ago

nikomatsakis commented 9 years ago

https://github.com/rust-lang/rust/issues/57563 | new meta tracking issue

Old content

Tracking issue for rust-lang/rfcs#911.

This issue has been closed in favor of more targeted issues:

Things to be done before stabilizing:

CTFE = https://en.wikipedia.org/wiki/Compile_time_function_execution

Munksgaard commented 9 years ago

Is this closed by #25609?

abonander commented 9 years ago

@Munksgaard That just adds support to the compiler AFAIK. There's a lot of functions in the stdlib that need to be changed to const fn and tested for breakage. I don't know what the progress is on that.

nodakai commented 9 years ago

I'm hoping this to be implemented on std::ptr::null() and null_mut() so that we can use them to initialize static mut *MyTypeWithDrop without resorting to 0usize as *mut _

rohel01 commented 9 years ago

EDIT: Removed since it was out of subject

glaebhoerl commented 9 years ago

To be clear, the question here is not primarily about the usefulness of the feature but rather regarding the best way to formulate it (or the best framework to formulate it in). See the RFC discussion.

aturon commented 9 years ago

This is now the tracking issue for eventual stabilization.

briansmith commented 8 years ago

https://github.com/rust-lang/rust/issues/29107 has been closed.

I disagree that "Integration with patterns", or any changes to the standard library should block this. This is very useful even without those changes, and those changes can be done later. In particular, I would like to start using const fn in my own code soon.

Accordingly, could the stabilization status of this be re-evaluated?

glaebhoerl commented 8 years ago

I don't doubt that const fn even in its current limited form would be useful functionality to have, but what I would really like, ideally before going further along this path, would be for those in favor of "the const fn approach" to think about and articulate their preferred endgame. If we just keep on incrementally adding useful-seeming functionality in the most obvious way, it seems very likely to me that we'll eventually end up copying more or less the entirety of C++'s constexpr design. Is that something we are comfortable with? Even if we say yes, I would much rather that we choose that path in a clear-eyed way, instead of backing into it with small steps over time, as the path of least resistance, until it has become inevitable.

(Given that the semantics of safe Rust code should be fully definable, it seems likely that eventually at least every function which doesn't (transitively) depend on unsafe should be able to be marked as const. And given that unsafe is supposed to be an implementation detail, I bet people will push for somehow loosening that restriction as well. I would much rather we looked abroad and tried to find a more cohesive, capable, and well-integrated story for staging and type-level computation.)

nikomatsakis commented 8 years ago

@glaebhoerl

I don't doubt that const fn even in its current limited form would be useful functionality to have, but what I would really like, ideally before going further along this path, would be for those in favor of "the const fn approach" to think about and articulate their preferred endgame...it seems very likely to me that we'll eventually end up copying more or less the entirety of C++'s constexpr design.

What I would personally like, even more than that, is that we have a fairly clear view on how we're going to implement it, and what portion of the language we are going to cover. That said, this is very closely related to support for associated constants or generics over integers in my mind.

@eddyb and I did some sketching recently on a scheme which could enable constant evaluation of a very broad swath of code. Basically lowering all constants to MIR and intrepreting it (in some cases, abstract interpreting, if there are generics you cannot yet evaluate, which is where things get most interesting to me).

However, while it seems like it would be fairly easy to support a very large fraction of the "builtin language", real code in practice hits up against the need to do memory allocation very quickly. In other words, you want to use Vec or some other container. And that's where this whole interpreting scheme starts to get more complicated to my mind.

That said, @glaebhoerl, I'd also love to hear you articulate your preferred alternative endgame. I think you sketched out some such thoughts in the const fn RFC, but I think it'd be good to hear it again, and in this context. :)

eddyb commented 8 years ago

The problem with allocation is having it escape into run-time. If we can somehow disallow crossing that compile-time/run-time barrier, then I believe we could have a working liballoc with const fn. It would be no harder to manage those kinds of allocations than would be to deal with byte-addressable values on an interpreted stack.

Alternatively, we could generate runtime code to allocate and fill in the values every time that barrier has to be passed, although I'm not sure what kind of usecases that has.

Keep in mind that even with full-fledged constexpr-like evaluation, const fn would still be pure: running it twice on 'static data would result in the exact same result and no side-effects.

glaebhoerl commented 8 years ago

@nikomatsakis If I had one I would have mentioned it. :) I mainly just see known unknowns. The whole thing with consts as part of the generics system was of course part of what I understood as being the C++ design. As far as having associated consts and const generic parameters, considering that we already have fixed-size arrays with consts as part of their type and would like to abstract over them, I would be surprised if there were a much better -- as opposed to merely more general -- way of doing it. The const fn part of things feels more separable and variable. It's easy to imagine taking things further and having things like const impls and const Trait bounds in generics, but I'm sure there is prior art for this sort of general thing which has already figured things out and we should try to find it.

Of the main use cases for the Rust language, the ones that primarily need low-level control, like kernels, seem reasonably well-served already, but another area where Rust could have lots of potential is things that primarily need high performance, and in that space powerful support (in some form) for staged computation (which const fn is already a very limited instance of) seems like it could be a game-changer. (Just in the last few weeks I came across two separate tweets by people who decided to switch from Rust to a language with better staging capabilities.) I'm not sure if any of the existing solutions in languages "close to us" -- C++'s constexpr, D's ad-hoc CTFE, our procedural macros -- really feel inspiring and powerful/complete enough for this sort of thing. (Procedural macros seem like a good thing to have, but more for abstraction and DSLs, not as much for performance-oriented code generation.)

As for what would be inspiring and good enough... I haven't seen it yet, and I'm not familiar enough with the whole space to know, precisely, where to look. Of course, per the above, we might want to at least glance at Julia and Terra, even if they seem like quite different languages from Rust in many ways. I know Oleg Kiselyov has done a lot of interesting work in this area. Tiark Rompf's work on Lancet and Lightweight Modular Staging for Scala seems definitely worth looking at. I recall seeing a presentation by @kmcallister at some point about what a dependently typed Rust might look like (which might at least be more general than sticking const everywhere), and I also recall seeing something from Oleg to the effect that types themselves are a form of staging (which feels natural considering the phase separation between compile- and runtime is a lot like stages)... lots of exciting potential connections in many different directions, which is why it'd feel like a missed opportunity if we were to just commit to the first solution which occurs to us. :)

(This was just a braindump and I've almost surely imperfectly characterized many things.)

briansmith commented 8 years ago

However, while it seems like it would be fairly easy to support a very large fraction of the "builtin language", real code in practice hits up against the need to do memory allocation very quickly. In other words, you want to use Vec or some other container. And that's where this whole interpreting scheme starts to get more complicated to my mind.

I disagree with that characterization of "real code in practice". I think there is big interest in Rust because it helps reduce the need for heap memory allocation. My code, in particular, makes a concrete effort to avoid heap allocation whenever possible.

Being able to do more than that would be nice but being able to construct static instances of non-trivial types with compiler-enforced invariants is essential. The C++ constexpr approach is extremely limiting, but it is more than what I need for my use cases: I need to provide a function that can construct an instance of type T with parameters x, y, and z such that the function guarantees that x, y, and z are valid (e.g., x < y && z > 0), such that the result can be a static variable, without the use of initialization code that runs at startup.

glaebhoerl commented 8 years ago

@briansmith FWIW another approach which has a chance of solving the same use cases would be if macros had privacy hygiene, which I believe (hope) we're planning to make them have.

briansmith commented 8 years ago

@briansmith FWIW another approach which has a chance of solving the same use cases would be if macros had privacy hygiene, which I believe (hope) we're planning to make them have.

I guess if you use the procedural macros then you can evaluate x < y && z > 0 at compile-time. But, it seems like it would be many, many months before procedural macros could be used in stable Rust, if they ever are. const fn is interesting because it can be enabled for stable Rust now, as far as I understand the state of things.

eddyb commented 8 years ago

@glaebhoerl I wouldn't hold my breath for strict hygiene, it's quite possible we're going to have an escape mechanism (just like real LISPs), so you may not want it for any kind of safety purposes.

nikomatsakis commented 8 years ago

@glaebhoerl there is also https://anydsl.github.io/, which even uses Rust-like syntax ;) they are basically targeting staged computation and in particular partial evaluation.

On Sat, Jan 16, 2016 at 6:29 PM, Eduard-Mihai Burtescu < notifications@github.com> wrote:

@glaebhoerl https://github.com/glaebhoerl I wouldn't hold my breath for strict hygiene, it's quite possible we're going to have an escape mechanism (just like real LISPs), so you may not want it for any kind of safety purposes.

— Reply to this email directly or view it on GitHub https://github.com/rust-lang/rust/issues/24111#issuecomment-172271960.

jethrogb commented 8 years ago

29525 should be fixed before stabilization

glaebhoerl commented 8 years ago

Given that the semantics of safe Rust code should be fully definable, it seems likely that eventually at least every function which doesn't (transitively) depend on unsafe should be able to be marked as const. And given that unsafe is supposed to be an implementation detail, I bet people will push for somehow loosening that restriction as well.

Just a thought: if we ever formally define Rust's memory model, then even unsafe code could potentially be safely and sensibly evaluated at compile-time by interpreting it abstractly/symbolically -- that is, use of raw pointers wouldn't turn into direct memory accesses like at runtime, but rather something (just as an example for illustration) like a lookup into a hashmap of allocated addresses, together with their types and values, or similar, with every step checked for validity -- so that any execution whose behavior is undefined would be strictly a compiler-reported error, instead of a security vulnerability in rustc. (This might also be connected to the situation w.r.t. handling isize and usize at compile-time symbolically or in a platform-dependent way.)

I'm not sure where that leaves us with respect to const fn. On the one hand, that would likely open up much more useful code to be available at compile time -- Box, Vec, Rc, anything using unsafe for performance optimization -- which is good. One arbitrary restriction fewer. On the other hand, the boundary for "what can possibly be a const fn" would now essentially be moved outwards to anything that doesn't involve the FFI. So what we'd actually be tracking in the type system, under the guise of constness, is what things (transitively) rely on the FFI and what things don't. And whether or not something uses the FFI is still something that people rightfully consider to be an internal implementation detail, and this restriction (unlike unsafe) really doesn't seem feasible to lift. And under this scenario you'd probably have far more fns being eligible for constness than ones which wouldn't.

So you'd still have constness revolving around an arbitrary, implementation-exposing restriction, and you'd also end up having to write const almost everywhere. That doesn't sound too appealing either...

eddyb commented 8 years ago

that is, use of raw pointers wouldn't turn into direct memory accesses like at runtime, but rather something ... like a lookup into a hashmap of allocated addresses,

@glaebhoerl Well, that is pretty much the model I described and which @tsion's miri is implementing.

I think the FFI distinction is very important because of purity, which is required for coherence. You couldn't even use GHC for Rust const fns because it has unsafePerformIO.

I don't like the const keyword too much myself which is why I am okay with const fn foo<T: Trait> instead of const fn foo<T: const Trait> (for requiring a const impl Trait for T).

Just like Sized, we probably have the wrong defaults, but I haven't seen any other proposals that can realistically work.

solson commented 8 years ago

@eddyb I think you meant to link to https://internals.rust-lang.org/t/mir-constant-evaluation/3143/31 (comment 31, not 11).

eddyb commented 8 years ago

@tsion Fixed, thanks!

MaikKlein commented 8 years ago

Please ignore this if I am completely off point.

The problem I see with this RFC is that as a user, you have to mark as many function const fn as possible because that will probably be the best practice. The same thing is happening currently in C++ with contexpr. I think this is just unnecessary verbosity.

D doesn't have const fn but it allows any function to be called at compile time ( with some exceptions ).

for example

// Standalone example.
struct Point { x: i32, y: i32 }

impl Point {
    fn new(x: i32, y: i32) -> Point {
        Point { x: x, y: y }
    }

    fn add(self, other: Point) -> Point {
        Point::new(self.x + other.x, self.y + other.y)
    }
}

const ORIGIN: Point = Point::new(0, 0); // works because 0, 0 are both known at compile time 
const ORIGIN2: Point = Point::new(0, 0); // ditto

const ANOTHER: Point = ORIGIN.add(ORIGIN2); // works because ORIGIN and ORIGIN2 are both const.
{
    let x: i32 = 42;
    let y: i32 = 24;
    const SOME_POINT: Point = Point::new(x, y); // Error: x and y are not known at compile time
}
{
    const x: i32 = 42;
    const y: i32 = 24;
    const SOME_POINT: Point = Point::new(x, y); // Works x and y are both known at compile time.
}

Note, I am not really a Rust user and I have only read the RFC a few minutes ago, so it is possible that I might have misunderstood something.

jethrogb commented 8 years ago

@MaikKlein there was a lot of discussion on CTFE in the RFC discussion

brson commented 7 years ago

I don't see any recent comments explaining the blockers here, and the op isn't very illuminating. What's the status. How can we move this across the finish line?

brson commented 7 years ago

This is used by Rocket: https://github.com/SergioBenitez/Rocket/issues/19#issuecomment-269052006

eddyb commented 7 years ago

See https://github.com/rust-lang/rust/issues/29646#issuecomment-271759986. Also we need to reconsider our position on explicitness since miri pushes the limit to "global side-effects" (@solson and @nikomatsakis were just talking about this on IRC).

oli-obk commented 7 years ago

The problem I see with this RFC is that as a user, you have to mark as many function const fn as possible because that will probably be the best practice.

While we could make arbitrary functions callable, if those functions access C code or statics we won't be able to compute them. As a solution I suggest a lint that will warn about public functions that could be const fn.

solson commented 7 years ago

I agree about the lint. It's similar to the existing built-in lints missing_docs, missing_debug_implementations, and missing_copy_implementations.

There's sort of a problem with having the lint on by default, though... it would warn about functions you explicitly don't want to be const, say, because you plan to later change the function such that it can't be const and don't want to commit your interface to const (removing const is a breaking change).

I guess #[allow(missing_const)] fn foo() {} might work in those cases?

solson commented 7 years ago

@eddyb @nikomatsakis My "removing const is a breaking change" point suggests we'll want to have the keyword after all, since it's a promise to downstream that the fn will remain const until the next major version.

It's going to be a shame how much const will need to be sprinkled through std and other libraries, but I don't see how you can avoid it, unless it was only required on public-facing items, and that seems like a confusing rule.

oli-obk commented 7 years ago

unless it was only required on public-facing items, and that seems like a confusing rule.

I like this one... I don't think it would be confusing. Your public interface is protected since you can't make a function not-const that is called by a const fn

Technically it would be better to annotate functions as notconst, because I expect there to be way more const fn than the other way around.

ssokolow commented 7 years ago

notconst would also be more consistent with Rust's design philosophy. (ie. "mut, not const")

solson commented 7 years ago

unless it was only required on public-facing items, and that seems like a confusing rule.

I like this one... I don't think it would be confusing.

I am flip flopping on this idea. It has its benefits (only think about const fn when making public interface decisions) but I thought of another way it could be confusing:

Your public interface is protected since you can't make a function not-const that is called by a const fn

This is true, and unfortunately it would imply that when a library author marks a public function const, then they are implicitly marking all functions transitively called by that function const as well, and there's a chance they're unintentionally marking functions they don't want to, thus preventing them from re-writing those internal functions using non-const features in the future.

I expect there to be way more const fn than the other way around.

I thought this way for a while, but it will only be true for pure Rust library crates. It won't be possible to make FFI-based fns const (even if they're only transitively FFI-based, which is a lot of stuff), so the sheer amount of const fn may not be quite as bad as you and I thought.

My current conclusion: Any non-explicit const fn seems problematic. There might just not be a good way to avoid writing the keyword a lot.

solson commented 7 years ago

Also, for the record, notconst would be a breaking change.

ssokolow commented 7 years ago

@solson A very good point.

eddyb commented 7 years ago

Keep in mind the keyword gets even hairier if you try to use it with trait methods. Restricting it to the trait definition isn't useful enough and annotating impls results in imperfect "const fn parametrim" rules.

nikomatsakis commented 7 years ago

I feel like this trade-off was pretty thoroughly discussed when we adopted const fn in the first place. I think @solson's analysis is also correct. I guess the only thing that has changed is that perhaps the percentage of constable fns has grown larger, but I don't think by enough to change the fundamental tradeoff here. It is going to be annoying to gradually have to add const fn into your public interfaces and so forth, but such is life.

eddyb commented 7 years ago

@nikomatsakis What's troubling me is the combination of these two facts:

Given that "global side-effects" is the main thing that prevents code from being const fn, isn't this the "effect system" that Rust used to have and got removed? Shouldn't we talk about "effect stability"? Seems similar to code assuming some library never panics IMO.

nikomatsakis commented 7 years ago

@eddyb absolutely const is an effect system and yes it does come with all the downsides that made us want to avoid them as much as possible... It is plausible that if we are going to endure the pain of adding in an effect system, we may want to consider some syntax that we can scale to other sorts of effects. As an example, we're paying a similar price with unsafe (also an effect), though I'm not sure that it makes sense to think about unifying those.

The fact that violations may occur dynamically seems like even more reason to make this opt-in though, no?

torkleyy commented 7 years ago

How about this:

In general, I think, const fns should only be used for constructors (new) or where absolutely necessary.

However, sometimes you may want to use other methods in order to conveniently create a constant. I think we could solve this problem for many cases by making constness the default but only for the defining module. This way, dependents cannot assume constness unless explicitly guaranteed with const, while still having the convenience to create constants with functions without making everything const.

eddyb commented 7 years ago

@torkleyy You can do that already by having helpers which are not exported.

mikeyhew commented 7 years ago

I don't see a strong argument that private helper functions shouldn't be implicitly const, when possible. I think @solson was saying that making const explicit, even for helper functions, forces the programmer to pause and consider whether they want to commit to that function being const. But if programmers are already required to think about that for public functions, isn't that enough? Wouldn't it be worth it not to have to write const everywhere?

durka commented 7 years ago

On IRC @eddyb proposed splitting this feature gate so that we could stabilize calls to const fns ahead of figuring out details of their declaration and bodies. Does that sound like a good idea?

SimonSapin commented 7 years ago

@durka That sounds great to me, as a Rust users who doesn’t know much about compiler internals.

nixpulvis commented 7 years ago

Excuse my lack of understanding here, but what does it mean to stabilize the call to a const function without stabilizing the declaration.

Are we saying that the compiler will somehow know what is and isn't constant through some means, but leave that part open for discussion/implementation for the time being?

How then can the calls be stabilized if the compiler might later change its mind on what is constant?

SimonSapin commented 7 years ago

@nixpulvis Some const fns already exist in the standard library, for example UnsafeCell::new. This proposal would make it allowed to call such functions in constant contexts, for example the initializer of a static item.

eddyb commented 7 years ago

@nixpulvis What I meant were calls to const fn functions defined by unstable-using code (such as the standard library), from constant contexts, not regular functions defined in stable Rust code.

eddyb commented 7 years ago

cc @rust-lang/lang on https://github.com/rust-lang/rust/issues/24111#issuecomment-310245900

SimonSapin commented 7 years ago

While I’m all in favor of stabilizing calls to const fns first if that can happen faster, it’s not clear to me what’s blocking stabilizing all of the const fn feature. What are the remaining concerns today? What would be a path to address them?

eddyb commented 7 years ago

@SimonSapin It's more that we're not clear the design for declaring const fns today scales well nor are we sure on the interactions between them and traits and how much flexibility there should be.

nikomatsakis commented 7 years ago

I think I'm inclined to stabilize uses of const fn. This seems like an ergonomic and expressiveness win and I still can't imagine a better way to handle compile-time constant evaluation than just being able to "write normal code".