search

rust-lang / rust

Empowering everyone to build reliable and efficient software.
https://www.rust-lang.org
Other
98.7k stars 12.75k forks source link

xLTO: Code from standard library does not partake in linker-based LTO (unless compiling with -Clto) #64124

Open michaelwoerister opened 5 years ago

michaelwoerister commented 5 years ago

xLTO works by emitting LLVM bitcode instead of machine code and then letting the linker run LLVM optimizations (at a point where also code from C/C++ is available for IPO). In the current setup, this means that code from the standard library does not partake in the final LTO step because it is pre-compiled to machine code.

This limitation can be lifted by compiling to a staticlib with -Clto because the fat LTO step that rustc does pulls in the compressed bitcode version of the standard library and then emits the unified bitcode file for further processing by the linker. However, this adds an additional fat LTO step into the compilation pipeline, which is very costly.

In theory it should be possible to make rustc emit libstd LLVM bitcode into the staticlib instead of the object files (if -Clinker-plugin-lto is specified). The bitcode is available and rustc knows how to decompress it. Then the linker could do its LTO step with the standard library included but without the additional cost.

NobodyXu commented 2 years ago

Hey, is this issue still alive?

It will be awesome if the libstd.rlib can contain LLVM bitcode so that LTO can be performed to reduce size of the binary.

Without LTO even a helloworld program would take 200KB or more.

bjorn3 commented 2 years ago

Libstd.rlib already contains LLVM bitcode, however it is only suitable for LTO performed by rustc (-Clto) rather than the linker (-Clinker-plugin-lto) which is what this issue is about. LTO alone can't reduce the size of hello world below 200kb as the binary among other things still includes code to print a backtrace on panics. If you want small rust binaries you can follow the instructions at https://github.com/johnthagen/min-sized-rust.

NobodyXu commented 2 years ago

Libstd.rlib already contains LLVM bitcode, however it is only suitable for LTO performed by rustc (-Clto) rather than the linker (-Clinker-plugin-lto) which is what this issue is about. LTO alone can't reduce the size of hello world below 200kb as the binary among other things still includes code to print a backtrace on panics. If you want small rust binaries you can follow the instructions at https://github.com/johnthagen/min-sized-rust.

Thank you for the explanation!

insilications commented 2 years ago

If you want std library to partake in linker-based LTO, do this. I've build the standard libraries with "-Clinker-plugin-lto=on -Ccodegen-units=1", so that all .rlib files contain the LLVM IR bitcode instead of the native object. I placed them all inside /usr/lib64/rustlib/x86_64-unknown-linux-gnu/lib/ and remove the previous ones. Following the guide (https://doc.rust-lang.org/nightly/rustc/linker-plugin-lto.html), you can them have all the benefits of LTO along with cross LTO, for all your program and all dependencies, including over the standard library as well.