rust-lang / compiler-team

A home for compiler team planning documents, meeting minutes, and other such things.
https://rust-lang.github.io/compiler-team/
Apache License 2.0
383 stars 67 forks source link

Partial compilation using MIR-only rlibs #738

Open davidtwco opened 4 months ago

davidtwco commented 4 months ago

Proposal

MIR-only rlibs are an idea that has been thrown around for years (rust-lang/rust#38913, rust-lang/rust#48373 and rust-lang/compiler-team#713 more recently). MIR-only rlibs have always been conceptualised as an artefact that you would use as a dependency to a downstream compilation, compiling anything you need from the MIR-only rlib on-demand. Instead, this MCP proposes "finishing off" a MIR-only rlib to produce a "full" rlib with object files, without needing the original crate source, only the MIR-only rlib.

@wesleywiser and I discussed using a mechanism like this for something like -Zbuild-std: Ship a MIR-only rlib for std, and -Zbuild-std would compile it to a regular rlib to be used as normal. Different compilation flags can be passed to the "finishing off" compilation (mostly codegen flags), enabling std to be built with sanitizers or a different mangling scheme, for example. Cross-compilation could just use the MIR-only rlib from the other target's rustup component. We could even ship MIR-only rlibs for tier three targets that users can "finish off" on their own. It wouldn't be helpful for cfgs, but we don't need to do everything at once. @saethlin suggested it might be useful for miri which just wants all of the MIR for std.

I have a very rough prototype implementation for this on my mir-only-rlib branch. I haven't done too much testing yet, it's capable of finishing-off a simple test case from its MIR-only rlib - it just proves that the idea is technically possible:

Ultimately, it's a little fragile in that you can use a query that requires something not in the cross-crate metadata in codegen and not know - we'd want to find a way to make this more robust. I haven't checked whether non-LLVM backends use more queries that aren't backed by the metadata.

Unlike MIR-only rlibs, implementation of this is quite different because it replaces what the compiler does with LOCAL_CRATE pretty significantly, so I wanted to check that the team agreed with this line of experimentation.

Mentors or Reviewers

@wesleywiser (interested stakeholder), @saethlin (previously interested in MIR-only rlibs)

Process

The main points of the Major Change Process are as follows:

You can read more about Major Change Proposals on forge.

Comments

This issue is not meant to be used for technical discussion. There is a Zulip stream for that. Use this issue to leave procedural comments, such as volunteering to review, indicating that you second the proposal (or third, etc), or raising a concern that you would like to be addressed.

rustbot commented 4 months ago

This issue is not meant to be used for technical discussion. There is a Zulip stream for that. Use this issue to leave procedural comments, such as volunteering to review, indicating that you second the proposal (or third, etc), or raising a concern that you would like to be addressed.

Concerns or objections to the proposal should be discussed on Zulip and formally registered here by adding a comment with the following syntax:

 @rustbot concern reason-for-concern 
 <description of the concern> 

Concerns can be lifted with:

 @rustbot resolve reason-for-concern 

See documentation at https://forge.rust-lang.org

cc @rust-lang/compiler @rust-lang/compiler-contributors