First-class compile time type manipulation

[cjhowedev]

Zig supports comptime, which is a keyword that allows for the evaluation of arbitrary expressions and entire functions at compile time. It is also used in Zig's implementation of generics, so that types can be manipulated at compile time to generate code. In particular, Zig supports the following compile time features:

• Treating types as first-class compile-time values
• Treating arguments as compile-time arguments as opposed to runtime arguments (that is, their value must be known at compile time)
• Generating structs/enums at compile time
• Emitting compiler errors from compile-time contexts, compile-time type validation
• Manually unroll loops (compile time for) and generate runtime code at compile time

Rust has a few existing features that address compile-time code evaluation:

• Const fns allow execution of a subset of Rust at compile time when a function is explicitly marked as const
  • Inline const is already in progress
• Procedural macros allow for execution of code at compile time to manipulate syntactical elements. They exist at compile time but work at a domain above the type system, so are limited in the kinds of analysis they can perform (especially around types)

These are a few of the goals that could be addressed with compile-time code evaluation and type manipulation:

• Variadic generics could be represented as a list of compile-time types without needing to worry about their runtime representation (including tuple byte layout issues in #376)
• Reflection can be supported without any runtime type representation necessary
• Manipulating types with regular code is a lot simpler than trying to generate code using procedural or even declarative macros in many cases
• Compile-time code can be run to validate preconditions necessary for generated code
• Optimized or specialized code can be generated using compile time code evaluation, without needing partial specialization, based on arbitrary conditions
• Contracts
• Tools (such as linters and IDEs) can run compile time code during development to ensure that types validate and produce correct code
• Compile-time code could access compiler features that aren't available at runtime (IE, ask the compiler to check for trait conformances)

I have some ideas for how this could be implemented, but ultimately I wanted to build a path towards variadic generics through compile-time code evaluation and generation. There are many features we would need to expand on first before we could work on many of the features listed above, mostly around making const and generics more powerful in Rust. And we would need a way to manipulate types in const fns and return them to be used as generic arguments.

What are the community's thoughts about this as a future direction for Rust?

[cjhowedev]

Here are the ideas I had for how we could go about implementing this feature by feature.

First, we could start by introducing const params to non-const functions that must be known at compile time. For a const fn all parameters must be known at compile time. For a non-const fn, some parameters may be const and others may be passed at runtime. I suspect this feature can be useful even with the existing const primitives we have today, even if limited. For example:

const myXs: [i32] = [1, 2, 5]

// Standard const fn (all variables and parameters const)
const fn my_const_fn(xs: [i32], y: i32) -> i32 {
  let sum: i32 = 0;
  for x in xs {
    sum += x;
  }
  return sum + y
}

// `my_const_fn(myXs, y)` not allowed unless y is const
// `my_const_fn(myXs, 6)` becomes `return (1 + 2 + 5) + 6 == 14` after compile time evaluation

// Non-const fn with const parameters
fn my_non_const_fn(const xs: [i32], y: i32) -> i32 {
  // Can use const parameters within const blocks
 return const {
    let mut sum: i32 = 0;
    for x in xs {
      sum += x;
    }
    sum
  } + y
}

// `my_non_const_fn(myXs, y)` is valid with non-const `y` and becomes `return (1 + 2 + 5) + y`
// `my_non_const_fn(myXs, 6)` becomes `return (1 + 2 + 5) + 6 == 14`, same as above

The next step would involve allowing the generation of runtime code within compile time contexts. This allows one to put runtime code within an inline const block:

fn runtime_in_const(const xs: [i32], ys: [i32]) -> i32 {
 const {
    for (i, x) in xs.enumerate() {
      runtime {
        if x > ys[i] {
          return x;
        }
      }
    }
  }
  return 0;
}

// runtime_in_const([1, 2], ys) would generate the following specialized function at runtime:
fn runtime_in_const_specialized(ys: [i32]) -> i32 {
  if 1 > ys[0] {
    return 1;
  }
  if 2 > ys[1] {
    return 2;
  }
  return 0;
}

The final step would be to reify types so they can be queried and manipulated in const contexts, and then provide a way to call a const fns on generic arguments to produce new type arguments. I'm still figuring out how this might look, but should start to resemble Zig.

[SichangHe]

Reference (not sure useful or not): https://github.com/soasis/rust/blob/ext/basic-reflection/library/core/src/introwospection.rs

[SichangHe]

This would be closer to what we have (except const generics only takes integers, bool and char):

// Non-const fn with const parameters
fn my_non_const_fn<const XS: &'static [i32]>(y: i32) -> i32 {
    let mut sum = 0;
    let mut index = 0;
    while index < XS.len() {
        sum += XS[index];
        index += 1;
    }
    sum + y
}

https://play.rust-lang.org/?version=stable&mode=debug&edition=2021&gist=376058daa0d54bc0956af2512e3f50ce

[SichangHe]

Blocking on https://github.com/rust-lang/rust/labels/F-generic_const_exprs https://github.com/rust-lang/rust/labels/F-adt_const_params, it seems.