Compiling FORM with --enable-static-link

[vsht]

Dear developers,

I remember having successfully created a statically linked FORM binary in the past (to have something that can be used on most machines/clusters without recompiling), but it seems that with the recent snapshot from the repository this doesn't work anymore.

I tried the following

./configure --disable-native --enable-static-link

on Fedora 37 with gcc 12.3.1 but apparently this fails because a static version of libmpfr is missing:

configure:7735: checking for static linking with g++ -static
configure:7760: g++ -o conftest    -static conftest.cpp  -lz -lmpfr -lgmp  >&5
/usr/bin/ld: cannot find -lmpfr: No such file or directory
collect2: error: ld returned 1 exit status
configure:7760: $? = 1
configure: failed program was:
...
configure:7765: result: no
configure:7735: checking for static linking with g++ -static-libgcc -static-libstdc++
configure:7760: g++ -o conftest    -static-libgcc -static-libstdc++ conftest.cpp  -lz -lmpfr -lgmp  >&5
configure:7760: $? = 0
configure:7762: result: yes
configure:7835: error: in `/media/Data/Projects/Other/form':
configure:7837: error: test for static linking failed. Give --disable-static-link if you want to build without static libraries.
See `config.log' for more details

In the past it was sufficient to install the static versions of libstdc++, glibc and zlib

sudo dnf install libstdc++-static glibc-static zlib-static

but since Fedora doesn't offer a static build of libmpfr, I'm wondering if there's perhaps some workaround one could come up with.

Cheers, Vlad

[tueda]

There is a similar problem for building Windows native binary (mingw-w64-x86_64) where the static version of the MPFR library is unavailable.

In such a case, an option would be to build a static library of MPFR (maybe GMP, too) from the source in such a way that it is "universal" (or "fat"). I haven't tried but vcpkg may be helpful for that.

Edit: indeed, our macOS binaries on the GitHub Release page use static libraries from Homebrew. Maybe a similar trick would work on Linux(?)

[vsht]

Many thanks for your reply! Creating a static build of MPFR turned out to be much simpler than I thought. Following this instruction

https://gapcoin-project.github.io/mining/2021/04/09/compiling-static-libgmp-and-libmpfr/

I succeeded in compiling a fat tform binary :) Surprisingly, in this case the --disable-native flag (cf. issue #457) seems to work as expected. My binary was created on a Ryzen 7 PRO 4750U that features avx and avx2 instruction sets, but the program runs also on older Xeon machines that have no avx support.

For the sake of completeness, here's the full instruction that I used on Fedora 37

Need to create static builds of mpfr and gmp first

export MPDIR=/media/Data/opt
cd ~/Downloads/BuildDir

curl -O https://gmplib.org/download/gmp/gmp-6.3.0.tar.xz
tar Jxf gmp-6.3.0.tar.xz

curl -O https://www.mpfr.org/mpfr-current/mpfr-4.2.0.tar.bz2
tar jxf mpfr-4.2.0.tar.bz2

cd gmp-6.3.0/
make clean

./configure --prefix=${MPDIR}/gmp --enable-static=yes --enable-shared=no --enable-cxx --with-pic 

make -j$(nproc)
make check
make install

cd ../mpfr-4.2.0
make clean

./configure --prefix=${MPDIR}/mpfr --enable-static=yes --disable-shared --with-pic --with-gmp=${MPDIR}/gmp

make -j$(nproc)
make check
make install

Installing the necessary libraries on Fedora

sudo dnf install libstdc++-static glibc-static zlib-static

Creating a universal fat form/tform binary (to be run from a FORM repo)

./configure --disable-native --enable-static-link --with-mpfr=/media/Data/opt/mpfr
make clean
make -j$(nproc)

[tueda]

Thanks for the try. I think there are 2 problems with your commands.

1. libgmp you built is not universal. The option--host (default: BUILD) or --enable-fat (default: no) is needed. See: https://gmplib.org/manual/Notes-for-Package-Builds. By the way, FORM does not use the C++ interface of GMP, so --enable-cxx is not needed for this purpose.
2. libgmp you built is not linked to FORM (unless you use the LIBRARY_PATH environment variable to specify its location). You also need --with-gmp=${MPDIR}/gmp in the configuration for FORM. Perhaps your binary was linked with the system GMP, which may be OK if the statically linked MPFR is ABI-compatible with it (you may try to configure your MPFR build with the system GMP). Indeed, I tried your commands (almost the same) on Docker:

   docker run -it --rm fedora:37

   
   export MYLIBDIR=/opt/tmp-mylib
   dnf install -y automake bzip2 diffutils gcc-c++ git glibc-static gmp-devel libstdc++-static make ruby rubygem-test-unit xz zlib-devel zlib-static

curl -O https://gmplib.org/download/gmp/gmp-6.3.0.tar.xz tar Jxf gmp-6.3.0.tar.xz pushd gmp-6.3.0 ./configure --prefix=$MYLIBDIR/gmp --enable-static --disable-shared --enable-cxx --with-pic make -j $(nproc) make check make install popd

curl -O https://www.mpfr.org/mpfr-current/mpfr-4.2.0.tar.bz2 tar jxf mpfr-4.2.0.tar.bz2 pushd mpfr-4.2.0 ./configure --prefix=$MYLIBDIR/mpfr --enable-static --disable-shared --with-pic --with-gmp=$MYLIBDIR/gmp make -j $(nproc) make check make install popd

git clone https://github.com/vermaseren/form pushd form autoreconf -i ./configure --disable-native --enable-static-link --with-mpfr=$MYLIBDIR/mpfr # fails make -j $(nproc) make check TEST_OPTS=--stat # fails due to #441

which gave me

configure: error: test for static linking failed. Give --disable-static-link if you want to build without static libraries.

because I had not installed the `gmp-static` package. Adding `--with-gmp`:
```bash
./configure --disable-native --enable-static-link --with-gmp=$MYLIBDIR/gmp --with-mpfr=$MYLIBDIR/mpfr

worked well and the subsequent compilation succeeded (make check failed, as expected).

[vsht]

Many thanks for your analysis. It's interesting that when running

make check TEST_OPTS=--stat

I actually get

Making check in doc
make[1]: Entering directory '/media/Data/Projects/Other/form/doc'
Making check in doxygen
make[2]: Entering directory '/media/Data/Projects/Other/form/doc/doxygen'
make[2]: Nothing to be done for 'check'.
make[2]: Leaving directory '/media/Data/Projects/Other/form/doc/doxygen'
Making check in manual
make[2]: Entering directory '/media/Data/Projects/Other/form/doc/manual'
make[2]: Nothing to be done for 'check'.
make[2]: Leaving directory '/media/Data/Projects/Other/form/doc/manual'
Making check in devref
make[2]: Entering directory '/media/Data/Projects/Other/form/doc/devref'
make[2]: Nothing to be done for 'check'.
make[2]: Leaving directory '/media/Data/Projects/Other/form/doc/devref'
make[2]: Entering directory '/media/Data/Projects/Other/form/doc'
make[2]: Nothing to be done for 'check-am'.
make[2]: Leaving directory '/media/Data/Projects/Other/form/doc'
make[1]: Leaving directory '/media/Data/Projects/Other/form/doc'
Making check in sources
make[1]: Entering directory '/media/Data/Projects/Other/form/sources'
/bin/sh "../scripts/git-version-gen.sh" -C "." -c -o version.h
make[1]: Leaving directory '/media/Data/Projects/Other/form/sources'
Making check in check
make[1]: Entering directory '/media/Data/Projects/Other/form/check'
make  check-TESTS
make[2]: Entering directory '/media/Data/Projects/Other/form/check'
FORM 4.1 (May 15 2023, v4.1-20131025-587-g8a37a42)
  0.053333 form units in 3.477 seconds (cpu time)
  corresponding to 55.22 form units per hour
PASS: benchmark-fu.sh
=============
1 test passed
=============
make[2]: Leaving directory '/media/Data/Projects/Other/form/check'
make[1]: Leaving directory '/media/Data/Projects/Other/form/check'
make[1]: Entering directory '/media/Data/Projects/Other/form'
make[1]: Leaving directory '/media/Data/Projects/Other/form'

But indeed, I had gmp-static in the version 6.2.1 installed, so perhaps it works just because it's ABI compatible with 6.3.0

Recompiling gmp and FORM using the flags you mentioned works as well.

I guess since Fedora ships a static flavor of libgmp one could probably just go with mpfr alone, but for the sake of completeness it's good to have a full instruction that might be useful for other distros as well.

[tueda]

It's interesting that when running

make check TEST_OPTS=--stat

I actually get

I think this is because you haven't installed Ruby, which is required to run the test suite; so only the benchmark test (benchmark-fu.sh) was invoked.

[vsht]

Actually I do have ruby installed

$ruby -v
ruby 3.1.4p223 (2023-03-30 revision 957bb7cb81) [x86_64-linux]

so I really don't know why the other part of the test wasn't invoked. If it is of relevance, here's the tform binary that I got when compiling things for the first time tform.zip

[tueda]

Can you see whether the configure script found Ruby's "test/unit" in config.log? As far as I remember, Fedora separately packaged rubygem-test-unit from ruby (see also the Fedora required packages in the Wiki).

[vsht]

Thanks, rubygem-test-unit was missing on my system.

BTW, it seems like not all tests pass in my case even though I'm using a vanilla version of form without any modifications

..................................
Finished in 30.401233705 seconds.
---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
230 tests, 849 assertions, 8 failures, 0 errors, 0 pendings, 0 omissions, 0 notifications
96.5217% passed
---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
7.57 tests/s, 27.93 assertions/s

Is this expected?

[tueda]

We know some test cases fail in the master branch. This is for development and is not guaranteed to work correctly. We need to understand what was changed from the stable version to fix the regressions.

[vsht]

I see, thanks for the clarification.