Static vs. dynamic linkage of packaged executable

[maxirmx]

Original requirement for tebako was to have it statically linked in order to make packaged executable portable.

After several attempts to implement this requirement and some research thi issue summarizes related findings.

Obstacles

1. It looks like static linking of Ruby core is not supported since version 2.5. Static linking may work but it is not a part of regression tests
2. Ruby static linking was designed to support platforms that do not have concept of loadable objects. So in case of static linking Ruby configuration scripts set -fvisibility=hidden compilation flag. However, native extensions normally reference some symbols back in the Ruby core and such symbols are not exported anymore because of hidden setting.
   Enforcing -fvisibility=default does not help, presumably due to some conditional defines in C code.
3. Building static applications does not eliminate references to system libraries. In particular

   Using 'dlopen' in statically linked applications requires at runtime the shared libraries from the glibc version used for linking
   Using 'getpwuid_r' in statically linked applications requires at runtime the shared libraries from the glibc version used for linking
   Using 'getaddrinfo' in statically linked applications requires at runtime the shared libraries from the glibc version used for linking
   Using 'gethostbyname' in statically linked applications requires at runtime the shared libraries from the glibc version used for linking

   (these are live warning from ld building tebako static executable)

Other implementations

Ruby-packer and ocra(??) create dynamically linked applications. Ruby-packer links statically libraries and objects that are likely to be missing or require upgrade or downgrade (for example, squashfs, ncurses, openssl, ...) but links dynamically to widely used system modules (pthread, dl or glibc)

According to @ronaldtse

So far we have been using ruby-packer and orca which uses dynamic linking, and don’t really have these issues

Proposed/implemented solution

When packaged solution is linked selection of static libraries is enforced in ruby-packer style - for the libraries that are likely to be missing or require upgrade or downgrade at end-user environment.

System libraries are linked dynamically:

        linux-vdso.so.1 (0x00007ffc3e386000)
        libpthread.so.0 => /lib/x86_64-linux-gnu/libpthread.so.0 (0x00007f0249dda000)
        libdl.so.2 => /lib/x86_64-linux-gnu/libdl.so.2 (0x00007f0249dd4000)
        libc.so.6 => /lib/x86_64-linux-gnu/libc.so.6 (0x00007f0249be2000)
        /lib64/ld-linux-x86-64.so.2 (0x00007f024b639000)
        libm.so.6 => /lib/x86_64-linux-gnu/libm.so.6 (0x00007f0249a93000)

The list above is managable and can be reduced further if more time is inveseted in experiments. Now the list allows to eliminated known issues with native extensions that were observed with static linking.

[maxirmx]

A note "just in case" If Ruby 2.7.4 is built statically compilation for bigdecimal extension fails.

The patch:

cp -f $PATCH_DIR/bigdecimal-patch.h $1/ext/bigdecimal/bigdecimal-patch.h
restore_and_save $1/ext/bigdecimal/bigdecimal.h
sed -i "s/#include <float.h>/#include <float.h>\n#include \"bigdecimal-patch.h\"\n/g" $1/ext/bigdecimal/bigdecimal.h

bigdecimal-patch.h

#ifndef HAVE_RB_SYM2STR
#define HAVE_RB_SYM2STR  1
#endif

#ifndef HAVE_RB_ARRAY_CONST_PTR
#define HAVE_RB_ARRAY_CONST_PTR 1
#endif

#ifndef HAVE_RB_RATIONAL_NUM
#define HAVE_RB_RATIONAL_NUM 1
#endif

#ifndef HAVE_RB_RATIONAL_DEN
#define HAVE_RB_RATIONAL_DEN 1
#endif

[ronaldtse]

Is the bigdecimal patch the only patch needed for static complication? Thanks.

[maxirmx]

Is the bigdecimal patch the only patch needed for static complication? Thanks. This is the only patch required for static compilation of core extensions.

Then comes the issue with native extensions referencing symbols in core Ruby and those symbols are not exported in case of static compilation. We observe it here: https://github.com/tamatebako/tebako/issues/41 For this issue I do not have a patch though I made several experiments and I believe I understand how to create a universal patch.

Then there is an issue with extensions that link dynamically the libraries that we link statically. For example, pthread. A project with such extension would compile, link and start but may crash. I think we observe it here: https://github.com/tamatebako/tebako/issues/38 For the issues of this nature I do not have any solution. I believe this will be additional requirement for extensions we can support and it seems both too vague and too strict at the same time.

[ronaldtse]

The notion of "static/dynamic" here is a little obscure because we can actually carry libraries in the tebako image?

For example, an extension that links dynamically to an optional system library, we could carry that system library in the tebako image.

[maxirmx]

https://developer.apple.com/forums/thread/655588

[ronaldtse]

The Apple situation may mean that the best approach is a custom compiled/patched Ruby, not dissimilar to how Linux distros all have their own compiled versions.

[maxirmx]

The issue with sassc (https://github.com/metanorma/packed-mn/pull/147) shows that it is even more complex

• Packaged application (patched Ruby, tebako image) is referencing limited set of shared objects on the host system. Line 1 on the diagram.

• Inside tebako image we may have "native extensions" . If a gem loads native extension using rubygems features this call is intercepted, a copy extension shared object is placed to host temp folder, all further calls to extension are routed to the copy of extension. Line 2 on the diagram.

• Extension itself and its copy may be dynamic and reference to the host libraries and/or Ruby entries exported by tebako image. Lines 3,4 on the diagram. If we build tebako image statically it has side effect. No symbols are exported and extension can not link to Ruby entries (i.e.: line 4 is broken). So we have to build tebako image itself as a shared object (~dynamically)

• If some ruby code uses external library through ffi or data file though sysetm calls like open (line 6 on the diagam), it does not work with ruby-packer and works with ocra. However, we have a set of hacks to support spevific gems (ffi, seven_zip_ruby, sassc, ...) These hacks extract shared libraries and data files to host temp folder and subclass supported gems in order to routes calls to copies of shared libraries and/or data files. Similar to point 2 above