LTO doesn't allow optimization by size

[Quuxplusone]

Bugzilla Link	PR38305
Status	NEW
Importance	P enhancement
Reported by	Mike Hommey (mh+llvm@glandium.org)
Reported on	2018-07-24 23:39:36 -0700
Last modified on	2018-07-25 17:19:40 -0700
Version	trunk
Hardware	PC Linux
CC	ditaliano@apple.com, george.burgess.iv@gmail.com, llvm-bugs@lists.llvm.org, peter@pcc.me.uk, tejohnson@google.com
Fixed by commit(s)
Attachments
Blocks
Blocked by
See also

$ clang -fuse-ld=gold -o t main.c -Os -flto=thin
/usr/bin/ld.gold: fatal error: Optimization level must be between 0 and 3
clang: error: linker command failed with exit code 1 (use -v to see invocation)

(Same with -Oz)

The error message is emitted by the llvm plugin, not gold (it's in
tools/lto/lto.cpp).

Interestingly, it fails at another level when using lld:

$ clang -fuse-ld=lld -o t main.c -Os -flto=thin
/usr/lib/llvm-6.0/bin/ld.lld: error: --plugin-opt: number expected, but got 's'
clang: error: linker command failed with exit code 1 (use -v to see invocation)

[Quuxplusone]

Basically the clang driver translates the -O argument to an argument to be passed to the linker (and the LTO library) for optimization. llvm doesn't really have an -OS LTO pipeline, hence the failure. We should either implement an -Os pipeline for LTO (hard) or not pass the argument down to the linker (and maybe fallback to a reasonable default?)

Cc:ing Teresa and Peter for opinions.

[Quuxplusone]

I chatted with Peter about this a while back. The conclusion, IIRC, was that
optimization levels for LTO should ideally be conceptually distinct from the
optimization levels supported by the compiler.

Moreover, it's not uncommon for us to LTO TUs compiled with different
optimization levels: someone declares that some subset of ${large_project} is
hot, compiles it with -O2, and compiles everything else with -Os or -Oz. Saying
"alright, well, this stuff I told you to optimize for size, I'd now like to
optimize for speed," or vice-versa, is pretty confusing to reason about, and
likely not what the user actually wants from the ThinLTO opt level.

My recollection is that we decided it's best to treat --lto-O$N as an
indication of how much *cross-TU* optimization that the user wanted to happen.
So, LTO's -O0 remains "none", -O1 becomes "minimal/mainly prefer size opts", -
O2 is what it is today, and -O3 becomes "-O2 but more," as is tradition. :)

So, tl;dr: the knob you're looking for may be spelled --lto-O1 in this brave
world. I haven't dived deeply enough to know how close we actually are to this
all yet, but ...

In general, my next goal in the LLVM world is to make {Thin,}LTO more size-
conscious when doing optimizations, so I'll take this bug in any case.

(FWIW, we already tag functions from -Os and -Oz TUs with optsize/minsize
attributes, so building with --lto-O2, assuming your TUs were all built with -
Os or -Oz, should still respect that today. Again, I don't yet have a deep
enough understanding of this all to say how perfectly we respect those attrs,
but hopefully soon I will. :) )

[Quuxplusone]

FWIW (as you can probably imagine reading my previous reply) I'm not a big fan of having -Os to be propagated to the linker, so I'm onboard with this plan, George.

[Quuxplusone]

I can confirm that building with -Os/-Oz and linking with -O2 works.