search

ros-infrastructure / buildfarm

Build scripts and notes for catkin debian build pipeline.
6 stars 15 forks source link

Default flags of trusty buildfarms? #197

Closed mikeferguson closed 10 years ago

mikeferguson commented 10 years ago

While tracking down an issue with MoveIt/geometry_shapes (https://github.com/ros-planning/geometric_shapes/issues/19), I started looking into the flags between Hydro/Precise and Indigo/Trusty. The bigger question here though is the default flags. The size of just about every library in debs is 3-4 times larger on Indigo/Trusty than it was with Hydro/Precise. A couple examples:

indigo:
-rw-r--r-- 1 root root  728K May  8 16:55 libtf2.so
-rw-r--r-- 1 root root  3.8M May  8 17:10 libroscpp.so

hydro:
-rw-r--r-- 1 root root  160K Mar  3 14:49 libtf2.so
-rw-r--r-- 1 root root  1.4M Mar  3 17:07 libroscpp.so

The number of symbols is also significantly larger:

ubr@ubr1:/opt/ros/indigo/lib$ readelf -Ws libroscpp.so | wc -l
13191
mikef@coruscant:/opt/ros/hydro/lib$ readelf -Ws libroscpp.so | wc -l
3094

ubr@ubr1:/opt/ros/indigo/lib$ readelf -Ws libtf2.so | wc -l
2567
mikef@coruscant:/opt/ros/hydro/lib$ readelf -Ws libtf2.so | wc -l
498

When building on precise/hydro (http://jenkins.ros.org/job/ros-hydro-geometric-shapes_binarydeb_precise_amd64/63/console) we get this for shapes.cpp:

/usr/lib/ccache/c++   -Dgeometric_shapes_EXPORTS -Dqh_QHpointer -g -O2 -fstack-protector --param=ssp-buffer-size=4 -Wformat -Wformat-security  -O3 -DNDEBUG -fPIC -I/tmp/buildd/ros-hydro-geometric-shapes-0.3.6-0precise-20140304-0007/include -isystem /usr/include/eigen3 -isystem /usr/include/assimp -isystem /opt/ros/hydro/include    -o CMakeFiles/geometric_shapes.dir/src/shapes.cpp.o -c /tmp/buildd/ros-hydro-geometric-shapes-0.3.6-0precise-20140304-0007/src/shapes.cpp

On trusty/indigo (http://jenkins.ros.org/job/ros-indigo-geometric-shapes_binarydeb_trusty_amd64/22/console):

/usr/lib/ccache/x86_64-linux-gnu-g++   -DASSIMP_UNIFIED_HEADER_NAMES -DGEOMETRIC_SHAPES_HAVE_QHULL_2011 -Dgeometric_shapes_EXPORTS -Dqh_QHpointer -fPIC -I/tmp/buildd/ros-indigo-geometric-shapes-0.3.8-1trusty-20140509-0010/include -isystem /usr/include/eigen3 -isystem /usr/include/assimp -isystem /opt/ros/indigo/include    -o CMakeFiles/geometric_shapes.dir/src/shapes.cpp.o -c /tmp/buildd/ros-indigo-geometric-shapes-0.3.8-1trusty-20140509-0010/src/shapes.cpp

This holds true with at least RVIZ, (searching robot.cpp in the buildlog quickly shows that there are no optimization flags on the indigo build).

To me, it looks like this is missing at the beginning of the trusty builds:

dpkg-buildpackage: export CFLAGS from dpkg-buildflags (origin: vendor): -g -O2 -fstack-protector --param=ssp-buffer-size=4 -Wformat -Wformat-security
dpkg-buildpackage: export CPPFLAGS from dpkg-buildflags (origin: vendor): -D_FORTIFY_SOURCE=2
dpkg-buildpackage: export CXXFLAGS from dpkg-buildflags (origin: vendor): -g -O2 -fstack-protector --param=ssp-buffer-size=4 -Wformat -Wformat-security
dpkg-buildpackage: export FFLAGS from dpkg-buildflags (origin: vendor): -g -O2
dpkg-buildpackage: export LDFLAGS from dpkg-buildflags (origin: vendor): -Wl,-Bsymbolic-functions -Wl,-z,relro
d

Which is concerning -- why is there no optimization?

dirk-thomas commented 10 years ago

I can confirm the issue. I compared the sizes of roscpp for Hydro and Indigo:

Quantal, Raring and Trusty do not pass any optimization flags.

dpkg-buildpackage seems to only set any compiler / linker flags on Precise:

dpkg-buildpackage: export CFLAGS from dpkg-buildflags (origin: vendor): -g -O2 -fstack-protector --param=ssp-buffer-size=4 -Wformat -Wformat-security
dpkg-buildpackage: export CPPFLAGS from dpkg-buildflags (origin: vendor): -D_FORTIFY_SOURCE=2
dpkg-buildpackage: export CXXFLAGS from dpkg-buildflags (origin: vendor): -g -O2 -fstack-protector --param=ssp-buffer-size=4 -Wformat -Wformat-security
dpkg-buildpackage: export FFLAGS from dpkg-buildflags (origin: vendor): -g -O2
dpkg-buildpackage: export LDFLAGS from dpkg-buildflags (origin: vendor): -Wl,-Bsymbolic-functions -Wl,-z,relro

More surprising is actually that Saucy is still doing optimized builds because dpkg-buildpackage does not that the flag on that platform either...

Anyway on my Trusty machine dpkg-buildflags returns the "correct* enabling optimized builds (similar to the flags above on Precise.

@tfoote The question would now be what is different in the pbuilder environments compared to a "normal" Ubuntu installation?

dirk-thomas commented 10 years ago

The man page of dpkg-buildflags (http://manpages.ubuntu.com/manpages/trusty/man1/dpkg-buildflags.1.html) states:

You should call  dpkg-buildflags  or  include  buildflags.mk  from  the
debian/rules file to obtain the needed build flags to pass to the build
system.  Note that older versions  of  dpkg-buildpackage  (before  dpkg
1.16.1)  exported  these  flags  automatically. However, you should not
rely on this, since this breaks manual invocation of debian/rules.

It looks like that bloom does currently not contain an invocation of dpkg-buildflags in the rules template: https://github.com/ros-infrastructure/bloom/blob/4952574611f37fec96c7d6f67bf9f6a5f8d2d3bf/bloom/generators/debian/templates/rules.em (@wjwwood FYI)

The version numbers of dpkg in the different Ubuntu versions (http://packages.ubuntu.com/search?keywords=dpkg&searchon=names&exact=1&suite=all&section=all) might support the assumption that this has just changed (if before dpkg 1.16.1 is interpreted as <= 1.16.1) which is supported by the changelog: http://changelogs.ubuntu.com/changelogs/pool/main/d/dpkg/dpkg_1.17.5ubuntu5.2/changelog:

Compiler flags are no longer exported
   Between   versions   1.14.17  and  1.16.1,  dpkg-buildpackage  exported
   compiler flags (CFLAGS, CXXFLAGS, FFLAGS, CPPFLAGS  and  LDFLAGS)  with
   values as returned by dpkg-buildflags. This is no longer the case.

Anyway it does not describe why Saucy is actually doing optimized builds:

mikeferguson commented 10 years ago

Fixed by https://github.com/ros-infrastructure/bloom/pull/278