Libraries built with lto and meson/ninja are larger than with configure/make

[juanitotc]

Using meson-0.43.0, ninja-1.8.2, gcc-7.2.0

Building glib-2.54.3 with lto as per below results in the following library sizes for libgio-2.0.so.0.5400.3 (stripped)

1,419,000 bytes with configure/make 1,525,688 bytes with meson/ninja

i.e. about 100kb or 10% larger with meson/ninja

from "ninja -v", it appears that lto is being used at link time, but maybe this isn't the case?

CC="gcc -flto -fuse-linker-plugin -mtune=generic -Os -pipe" CXX="g++ -flto -fuse-linker-plugin -mtune=generic -Os -pipe -fno-exceptions -fno-rtti" meson --prefix=/usr/local --buildtype=plain -Dwith-pcre=system -Dwith-docs=no -Denable-libmount=no ..

ninja [fails at link ninja: build stopped: subcommand failed unless locale set to UTF8] [1m 17.80s]
sudo ninja install

CC="gcc -flto -fuse-linker-plugin -mtune=generic -Os -pipe" CXX="g++ -flto -fuse-linker-plugin -mtune=generic -Os -pipe -fno-exceptions -fno-rtti" ./configure --prefix=/usr/local --localstatedir=/var --disable-static --with-pcre=system --disable-libmount

find . -name Makefile -type f -exec sed -i 's/-g -O2//g' {} \;

make [1m 31.96s]
sudo make install

[Salamandar]

First of all: You should pass CFlags with the CFLAGS envvar, not CC :) Then, you can build with verbose flags to get the CFlags actually passed to the compiler/linker.

• ninja -v
• V=1 make That'll help everyone debugging this.

BTW : When porting Gimp to Meson I also found bigger libraries. See here: https://git.gnome.org/browse/gimp/tree/meson_todo.md?h=meson#n61

[juanitotc]

Using CC/CXX or CFLAGS/CXXFLAGS makes absolutely no difference to the size of the libs built - I usually use CC/CXX to ensure that lto is used at the final link stage.

Anyway, build logs attached (tar.gz not supported) log_glib_ninja.zip log_glib_make.zip

[nirbheek]

This is very interesting. Can you try to reduce the issue down a bit? The first step would be comparing the compiler arguments used by both, and then making educated guesses and tweaking them.

[juanitotc]

OK, but the compiler arguments are identical, no?

[nirbheek]

Most are, but some aren't:

make:

gcc -DHAVE_CONFIG_H -I. -I.. -DG_LOG_DOMAIN=\"GLib-GIO\" -I.. -I../glib -I../glib -I.. -I../gmodule -DG_DISABLE_CAST_CHECKS -DGIO_COMPILATION -DGIO_MODULE_DIR=\"/usr/local/lib/gio/modules\" -pthread -Wall -Wstrict-prototypes -Werror=declaration-after-statement -Werror=missing-prototypes -Werror=implicit-function-declaration -Werror=pointer-arith -Werror=init-self -Werror=format=2 -Werror=missing-include-dirs -fvisibility=hidden -flto -fuse-linker-plugin -mtune=generic -Os -pipe -MT libgio_2_0_la-gfile.lo -MD -MP -MF .deps/libgio_2_0_la-gfile.Tpo -c gfile.c -fPIC -DPIC -o .libs/libgio_2_0_la-gfile.o

ninja:

cc -Igio/gio-2.0@sha -Igio -I../gio -I. -I../ -Iglib -I../glib -Igobject -I../gobject -Igmodule -I../gmodule -fdiagnostics-color=always -pipe -D_FILE_OFFSET_BITS=64 -std=gnu89 -D_GNU_SOURCE -flto -fuse-linker-plugin -mtune=generic -Os -fPIC '-DG_LOG_DOMAIN="GLib-GIO"' -DGIO_COMPILATION '-DGIO_MODULE_DIR="/usr/local/lib/gio/modules"' -fvisibility=hidden -MMD -MQ 'gio/gio-2.0@sha/gfile.c.o' -MF 'gio/gio-2.0@sha/gfile.c.o.d' -o 'gio/gio-2.0@sha/gfile.c.o' -c ../gio/gfile.c

I can see at least -std=gnu89, -D_GNU_SOURCE and -DHAVE_CONFIG_H are different. This might be relevant or it might not be. You should also compare the size before and after installation, and also check the link command.

[juanitotc]

I tried various combinations of -std=gnu89, -D_GNU_SOURCE and -DHAVE_CONFIG_H, but I did not see a change in size before or after installation.

I haven't found a way to manually link with meson/ninja - with make you can paste the command after the objects are built.

[nirbheek]

The link command is printed by ninja, and you can run it from inside the build directory just fine.

[tp-m]

What about the individual object files? Are they larger already?

[juanitotc]

I compared the object files (hopefully the correct ones) - in general the meson/ninja files are slightly larger than the configure/make files. Here's some of them:

48,320 libgio_2_0_la-gaction.o 59,768 libgio_2_0_la-gactiongroup.o 60,128 libgio_2_0_la-gactiongroupexporter.o 23,232 libgio_2_0_la-gactionmap.o 111,376 libgio_2_0_la-gappinfo.o 220,024 libgio_2_0_la-gapplication.o 64,712 libgio_2_0_la-gapplicationcommandline.o 84,920 libgio_2_0_la-gapplicationimpl-dbus.o

672,480

48,336 gaction.c.o 59,912 gactiongroup.c.o 60,368 gactiongroupexporter.c.o 23,408 gactionmap.c.o 112,672 gappinfo.c.o 219,152 gapplication.c.o 64,888 gapplicationcommandline.c.o 86,592 gapplicationimpl-dbus.c.o

675,328

I also tried compiling with both without lto - at 1,468,224 configure/make comes out at 148k or 10% smaller than meson/ninja at 1,615,960, which is in line with the size difference when lto is used.

[nirbheek]

Are you sure the same compiler is used for both autotools and meson? i.e., is cc the same as gcc? Since object files also have a size difference, it should be easier to figure out what's causing that difference. Meson is not involved in compilation at all, and Ninja just spawns cc with the specified arguments.

[juanitotc]

Yes, I'm sure:

$ which cc /usr/local/bin/cc $ ls -l /usr/local/bin/cc lrwxrwxrwx 1 root root 3 Feb 20 18:57 /usr/local/bin/cc -> gcc

Why not try for yourself, glib-2.54.3 only takes a couple of minutes to compile..

[orbea]

I looked at this with samurai which is perhaps a smaller and more simple test case.

https://github.com/michaelforney/samurai

Without -flto the combined size of the object files is the same, but some object files are consistently a different size.

samu

[1/13] clang -MD -MF util.o.d -Wall -Wextra -std=c99 -pedantic -c -o util.o util.c
[2/13] clang -MD -MF tree.o.d -Wall -Wextra -std=c99 -pedantic -c -o tree.o tree.c
[3/13] clang -MD -MF tool.o.d -Wall -Wextra -std=c99 -pedantic -c -o tool.o tool.c
[4/13] clang -MD -MF scan.o.d -Wall -Wextra -std=c99 -pedantic -c -o scan.o scan.c
[5/13] clang -MD -MF samu.o.d -Wall -Wextra -std=c99 -pedantic -c -o samu.o samu.c
[6/13] clang -MD -MF parse.o.d -Wall -Wextra -std=c99 -pedantic -c -o parse.o parse.c
[7/13] clang -MD -MF log.o.d -Wall -Wextra -std=c99 -pedantic -c -o log.o log.c
[8/13] clang -MD -MF htab.o.d -Wall -Wextra -std=c99 -pedantic -c -o htab.o htab.c
[9/13] clang -MD -MF graph.o.d -Wall -Wextra -std=c99 -pedantic -c -o graph.o graph.c
[10/13] clang -MD -MF env.o.d -Wall -Wextra -std=c99 -pedantic -c -o env.o env.c
[11/13] clang -MD -MF deps.o.d -Wall -Wextra -std=c99 -pedantic -c -o deps.o deps.c
[12/13] clang -MD -MF build.o.d -Wall -Wextra -std=c99 -pedantic -c -o build.o build.c
[13/13] clang  -o samu build.o deps.o env.o graph.o htab.o log.o parse.o samu.o scan.o tool.o tree.o util.o

360K    samurai/.git
16K samurai/.builds
4.0K    samurai/.github
764K    samurai/

16K build.o
16K deps.o
8.0K    env.o
8.0K    graph.o
8.0K    htab.o
8.0K    log.o
12K parse.o
16K samu.o
12K scan.o
8.0K    tool.o
4.0K    tree.o
8.0K    util.o
124K    total

make

/usr/bin/clang -O -Wall -Wextra -std=c99 -pedantic -c -o build.o build.c
/usr/bin/clang -O -Wall -Wextra -std=c99 -pedantic -c -o env.o env.c
/usr/bin/clang -O -Wall -Wextra -std=c99 -pedantic -c -o deps.o deps.c
/usr/bin/clang -O -Wall -Wextra -std=c99 -pedantic -c -o graph.o graph.c
/usr/bin/clang -O -Wall -Wextra -std=c99 -pedantic -c -o htab.o htab.c
/usr/bin/clang -O -Wall -Wextra -std=c99 -pedantic -c -o log.o log.c
/usr/bin/clang -O -Wall -Wextra -std=c99 -pedantic -c -o parse.o parse.c
/usr/bin/clang -O -Wall -Wextra -std=c99 -pedantic -c -o samu.o samu.c
/usr/bin/clang -O -Wall -Wextra -std=c99 -pedantic -c -o scan.o scan.c
/usr/bin/clang -O -Wall -Wextra -std=c99 -pedantic -c -o tool.o tool.c
/usr/bin/clang -O -Wall -Wextra -std=c99 -pedantic -c -o tree.o tree.c
/usr/bin/clang -O -Wall -Wextra -std=c99 -pedantic -c -o util.o util.c
/usr/bin/clang  -o samu build.o env.o deps.o graph.o htab.o log.o parse.o samu.o scan.o tool.o tree.o util.o

360K    samurai/.git
16K samurai/.builds
4.0K    samurai/.github
744K    samurai/

16K build.o
16K deps.o
8.0K    env.o
8.0K    graph.o
8.0K    htab.o
8.0K    log.o
8.0K    parse.o
12K samu.o
20K scan.o
8.0K    tool.o
4.0K    tree.o
8.0K    util.o
124K    total

Notice the parse.o, samu.o and scan.o files where the first two are smaller with make and the last one is smaller with samu.

samu with lto

[1/13] clang -flto -MD -MF util.o.d -Wall -Wextra -std=c99 -pedantic -c -o util.o util.c
[2/13] clang -flto -MD -MF tree.o.d -Wall -Wextra -std=c99 -pedantic -c -o tree.o tree.c
[3/13] clang -flto -MD -MF tool.o.d -Wall -Wextra -std=c99 -pedantic -c -o tool.o tool.c
[4/13] clang -flto -MD -MF scan.o.d -Wall -Wextra -std=c99 -pedantic -c -o scan.o scan.c
[5/13] clang -flto -MD -MF samu.o.d -Wall -Wextra -std=c99 -pedantic -c -o samu.o samu.c
[6/13] clang -flto -MD -MF parse.o.d -Wall -Wextra -std=c99 -pedantic -c -o parse.o parse.c
[7/13] clang -flto -MD -MF log.o.d -Wall -Wextra -std=c99 -pedantic -c -o log.o log.c
[8/13] clang -flto -MD -MF htab.o.d -Wall -Wextra -std=c99 -pedantic -c -o htab.o htab.c
[9/13] clang -flto -MD -MF graph.o.d -Wall -Wextra -std=c99 -pedantic -c -o graph.o graph.c
[10/13] clang -flto -MD -MF env.o.d -Wall -Wextra -std=c99 -pedantic -c -o env.o env.c
[11/13] clang -flto -MD -MF deps.o.d -Wall -Wextra -std=c99 -pedantic -c -o deps.o deps.c
[12/13] clang -flto -MD -MF build.o.d -Wall -Wextra -std=c99 -pedantic -c -o build.o build.c
[13/13] clang -flto  -o samu build.o deps.o env.o graph.o htab.o log.o parse.o samu.o scan.o tool.o tree.o util.o

360K    samurai/.git
16K samurai/.builds
4.0K    samurai/.github
800K    samurai/

20K build.o
20K deps.o
12K env.o
12K graph.o
12K htab.o
12K log.o
12K parse.o
16K samu.o
16K scan.o
12K tool.o
8.0K    tree.o
12K util.o
164K    total

make with lto

clang -flto -O -Wall -Wextra -std=c99 -pedantic -c -o build.o build.c
clang -flto -O -Wall -Wextra -std=c99 -pedantic -c -o env.o env.c
clang -flto -O -Wall -Wextra -std=c99 -pedantic -c -o deps.o deps.c
clang -flto -O -Wall -Wextra -std=c99 -pedantic -c -o graph.o graph.c
clang -flto -O -Wall -Wextra -std=c99 -pedantic -c -o htab.o htab.c
clang -flto -O -Wall -Wextra -std=c99 -pedantic -c -o log.o log.c
clang -flto -O -Wall -Wextra -std=c99 -pedantic -c -o parse.o parse.c
clang -flto -O -Wall -Wextra -std=c99 -pedantic -c -o samu.o samu.c
clang -flto -O -Wall -Wextra -std=c99 -pedantic -c -o scan.o scan.c
clang -flto -O -Wall -Wextra -std=c99 -pedantic -c -o tool.o tool.c
clang -flto -O -Wall -Wextra -std=c99 -pedantic -c -o tree.o tree.c
clang -flto -O -Wall -Wextra -std=c99 -pedantic -c -o util.o util.c
clang -flto  -o samu build.o env.o deps.o graph.o htab.o log.o parse.o samu.o scan.o tool.o tree.o util.o

360K    samurai/.git
16K samurai/.builds
4.0K    samurai/.github
788K    samurai/

20K build.o
20K deps.o
12K env.o
12K graph.o
12K htab.o
12K log.o
12K parse.o
16K samu.o
28K scan.o
12K tool.o
8.0K    tree.o
12K util.o
176K    total

With -flto the only difference seems to be scan.o which is bigger with make.

I narrowed down the difference in size to the -MF, I'm not sure how much of this issue is related to meson or even broken?

[xclaesse]

AFAIK, -MD -MF parse.o.dshould not affect the compiled object (or that would be a clang bug).

The other difference is -O, which could be relevant. What's the optimization level when giving -O without a number? Could be different than not giving -O at all.

[xclaesse]

https://clang.llvm.org/docs/ClangCommandLineReference.html#optimization-level

So -O is same as -O2 and AFAIK omitting that arg is same as -O0. That would definitely explain the difference you see.

[orbea]

Yes you are right about the -O, I am not sure what I saw earlier with -MF as I can't reproduce it anymore.

clang -O0 -Wall -Wextra -std=c99 -pedantic -c -o build.o build.c
clang -O0 -Wall -Wextra -std=c99 -pedantic -c -o env.o env.c
clang -O0 -Wall -Wextra -std=c99 -pedantic -c -o deps.o deps.c
clang -O0 -Wall -Wextra -std=c99 -pedantic -c -o graph.o graph.c
clang -O0 -Wall -Wextra -std=c99 -pedantic -c -o htab.o htab.c
clang -O0 -Wall -Wextra -std=c99 -pedantic -c -o log.o log.c
clang -O0 -Wall -Wextra -std=c99 -pedantic -c -o parse.o parse.c
clang -O0 -Wall -Wextra -std=c99 -pedantic -c -o samu.o samu.c
clang -O0 -Wall -Wextra -std=c99 -pedantic -c -o scan.o scan.c
clang -O0 -Wall -Wextra -std=c99 -pedantic -c -o tool.o tool.c
clang -O0 -Wall -Wextra -std=c99 -pedantic -c -o tree.o tree.c
clang -O0 -Wall -Wextra -std=c99 -pedantic -c -o util.o util.c
clang  -o samu build.o env.o deps.o graph.o htab.o log.o parse.o samu.o scan.o tool.o tree.o util.o

364K    samurai/.git
16K samurai/.builds
4.0K    samurai/.github
756K    samurai/

16K build.o
16K deps.o
8.0K    env.o
8.0K    graph.o
8.0K    htab.o
8.0K    log.o
12K parse.o
16K samu.o
12K scan.o
8.0K    tool.o
4.0K    tree.o
8.0K    util.o
124K    total

[1/13] clang -O0 -MD -MF util.o.d -Wall -Wextra -std=c99 -pedantic -c -o util.o util.c
[2/13] clang -O0 -MD -MF tree.o.d -Wall -Wextra -std=c99 -pedantic -c -o tree.o tree.c
[3/13] clang -O0 -MD -MF tool.o.d -Wall -Wextra -std=c99 -pedantic -c -o tool.o tool.c
[4/13] clang -O0 -MD -MF scan.o.d -Wall -Wextra -std=c99 -pedantic -c -o scan.o scan.c
[5/13] clang -O0 -MD -MF samu.o.d -Wall -Wextra -std=c99 -pedantic -c -o samu.o samu.c
[6/13] clang -O0 -MD -MF parse.o.d -Wall -Wextra -std=c99 -pedantic -c -o parse.o parse.c
[7/13] clang -O0 -MD -MF log.o.d -Wall -Wextra -std=c99 -pedantic -c -o log.o log.c
[8/13] clang -O0 -MD -MF htab.o.d -Wall -Wextra -std=c99 -pedantic -c -o htab.o htab.c
[9/13] clang -O0 -MD -MF graph.o.d -Wall -Wextra -std=c99 -pedantic -c -o graph.o graph.c
[10/13] clang -O0 -MD -MF env.o.d -Wall -Wextra -std=c99 -pedantic -c -o env.o env.c
[11/13] clang -O0 -MD -MF deps.o.d -Wall -Wextra -std=c99 -pedantic -c -o deps.o deps.c
[12/13] clang -O0 -MD -MF build.o.d -Wall -Wextra -std=c99 -pedantic -c -o build.o build.c
[13/13] clang  -o samu build.o deps.o env.o graph.o htab.o log.o parse.o samu.o scan.o tool.o tree.o util.o

364K    samurai/.git
16K samurai/.builds
4.0K    samurai/.github
768K    samurai/

16K build.o
16K deps.o
8.0K    env.o
8.0K    graph.o
8.0K    htab.o
8.0K    log.o
12K parse.o
16K samu.o
12K scan.o
8.0K    tool.o
4.0K    tree.o
8.0K    util.o
124K    total

[JoonasPitk]

I happened to find this issue after trying to figure out why a game's launcher ran through Proton freezes the system when pixman version number is 0.43.0. See the issue on pixman GitLab here. I built commit 396e1a76edbbe96b6584a80bc850243ed0f8a91d (last of 0.42.3), commit 6c2e4a0dd9d1c84f501f9b764f08d258e03b3357 (bump to 0.43.0), and commit 8e4be8c2dbd605d54c79484d9f5d8befd42bd6f3 (bump to version 0.43.1) with plain meson setup buildX/ and ninja -C buildX. Why does this build a different size binary just because the version number happens to be 0.43.0?

[eli-schwartz]

Meson doesn't build anything, the compiler builds it.

This is an important distinction, because it means you can compare the build.ninja file before and after changing the version number, and check how the compiler is being called.

I happened to find this issue after trying to figure out why a game's launcher ran through Proton freezes the system when pixman version number is 0.43.0.

Sadly this issue has no relationship whatsoever to your issue. This issue is about different behavior when comparing meson to autotools. Your issue is about different behavior when comparing meson to meson.