Quuxplusone / LLVMBugzillaTest

0 stars 0 forks source link

Size of built debug-mode binaries is enormous #34485

Open Quuxplusone opened 6 years ago

Quuxplusone commented 6 years ago
Bugzilla Link PR35512
Status NEW
Importance P enhancement
Reported by Gordon Keiser (codeman.consulting@gmail.com)
Reported on 2017-12-03 23:39:04 -0800
Last modified on 2017-12-05 11:51:01 -0800
Version trunk
Hardware PC Linux
CC clang@martinien.de, dblaikie@gmail.com, llvm-bugs@lists.llvm.org, spatel+llvm@rotateright.com
Fixed by commit(s)
Attachments
Blocks
Blocked by
See also

This may be something to do with any number of things other than CMake, but that's what I'm generating the build scripts with so I'm putting this here for now... the size of llvm binaries has somehow become enormous on Ubuntu. Here's Clang, for example:

-rwxr-xr-x 1 root root 1824M Dec 3 22:42 /usr/local/bin/clang-6.0

LLVM binaries have never been exactly small in debug mode, but this looks new to me. Is there some type of dwarf generation issue or flag I can use to prevent this? As of now make followed by make install on an 80GB VM disk (which had always been enough for quick builds and is nearly unused otherwise) almost filled the disk starting from having 70GB free. strip -g on clang-6.0 reduced this size to:

-rwxr-xr-x 1 root root 299M Dec 3 23:29 /usr/local/bin/clang-6.0

Which is still rather enormous although acceptable, but defeats the purpose of a debug build in the first place. I thought I built this as a shared library version, but it seems to just link in all the debug info from the static libs anyway which also defeats the purpose of that.

Is there some flag or trick I'm missing to keep the debug info from inflating size this much? Can it be produced externally with some flag or another? Build was with a relatively recent GCC, it built debug by default with make -j8. Those file sizes also limit the ability to run VM disks from regular hard drives because the disk activity is the limiting factor, hence the small VM disk size on a fast SSD to keep up.

Is this even normal?

I'm marking as an enhancement because if there's something we can add to CMake to help stop this behavior? Am I missing a flag somewhere? My windows release build ended up with a 50M clang++.exe with the same targets (ALL), and RTTI and EH turned on in addition to that, so this seems to be something more linux specific or I'm forgetting something.

Quuxplusone commented 6 years ago
This might be better as a discussion on the mailing list than a bug as such.

You mentioned it has "become enormous" - do you have a revision at which it was
substantially smaller?

Are you sure you're getting a shared library build? Not many people use them (&
I don't) so I could be wrong, but yes, I'd expect smaller final binaries if
they just reference a bunch of shared libraries.

Clang's debug info is substantially smaller than GCC's, so you could try
building your Clang with Clang rather than with GCC

You could also try using Split DWARF (there's a flag for it in CMake -
LLVM_USE_SPLIT_DWARF) which avoids having the debug info in the .o files and
avoids copying it into the final executable so may reduce the overall disk
usage footprint of a build.

Windows uses separate debug info (somewhat like the Split DWARF mentioned
above) files (PDBs) so its binaries are smaller but the actual debug info/total
build disk usage footpring may still be large. Unix/DWARF debugging has
historically been in the object file and linked into the final executable -
perhaps for ease of generation and distribution (if you send a debug-having
binary to somewhere, you have the debug info - it's not in separate files you
have to handle in addition)
Quuxplusone commented 6 years ago
A thread popped up about this on the mailing list before I could post one
actually.  Apparently LLVM is missing some flags involving dwarf sections and
64 bit dwarf info that cause the file sizes to blow up.  Their thread was about
*our* dwarf output (this was a GCC build) but the flags and suggestions in the
thread seem like a logical first step towards fixing this.

I'm still convinced the failure to use .so files instead of static linked files
is an error on my part somewhere or something that can be fixed in CMake which
will also help the issue I'm seeing.  The build logs indicate that he shared
libaries are all created and some projects link with them, but the static
libraries link with everything.    This seems like a configure error on my part
or something that I can debug.

As I mentioned on the list I'm leaving this open for now, especially because we
have a you can check out the thread "[llvm-dev] [RFC] - Deduplication of debug
information in linkers (LLD)."

The primary two pieces of info from the thread are from George Rimar:

-fdebug-types-section, -fno-debug-types-section
Place debug types in their own section (ELF Only)
gcc's description is here: https://gcc.gnu.org/onlinedocs/gcc-
6.4.0/gcc/Debugging-Options.html#Debugging-Options.

>This flag is disabled by default. I compared clang binaries to see the
difference
>with and without the linker side optimisation.
>1) Clang built with -g has size of 1.7 GB, .debug_info section size is 894.5
Mb.
>2) Clang built with -g -fdebug-types-section has size of 1.0 GB.
>   .debug_types size is 26.267 MB, .debug_info size is 227.7 MB.

As for revisions, I couldn't pinpoint one.  I know it was substantially smaller
than that in the 2.x-3.x era when I was building it multiple times daily.  We
usually built in release but debug builds weren't as bad.  Cutting it down to
1GB from nearly 2GB with a flag is a great improvement for me if it works as
well.

I vote for adding  it to CMake debug builds by default if it doesn't cause any
errors in the actual code.

And yeah, the PDB thing on windows is a double-edged sword as well.  I made the
mistake of downloading the full PDB set for the Windows kernel when I installed
the driver development kit once and my hard drive hated me for it.  :-)
Quuxplusone commented 6 years ago

Honestly I'd rather see multi-terabyte SSDs become very cheap and eliminate this problem. I do a lot of my development work in a VM and even with all the virtualization technologies enabled, the latency of a HDD combined with a virtual disk's added delay from being bad with the underlying disk topology (Hyper-V doesn't have that issue, but I can't get linux to properly handle the video drivers on it. Anyway, that's off topic but at least someone got much closer to a solution in some weird case of synchronicity. I like the list but can't keep up with the volume most of the time, so for things like that I'll file here. For things I can fix quickly I generally just fix them, but I'm still catching up to 3 years worth of changes so I've been sort of compiling a list of things I know I can reasonably fix on the bugzilla lately.

Quuxplusone commented 6 years ago

FWIW - type units may hinder debug size if you're using Split DWARF (which may be a bigger win in terms of link/compile time, and probably in terms of total disk usage for a build), since they have some overhead to make the types sufficiently isolated/standalone to be stripped/deduplicated by the linker.

I'd generally encourage use of Split DWARF if that's viable for you.