Add support for --host flag in configure script

[fawazahmed0]

Hi, I would like to cross compile tilde as static binaries to different architectures (such as aarch64,mips etc) and keep them at git repo for sharing. could you add support for ./configure --host flag. Thanks

[gphalkes]

This will be somewhat complicated, as the current build system doesn't have separation for the build system and the target system. As a few utils are built that are run on the build system, this separation is necessary to make this work.

[fawazahmed0]

@gphalkes , I thought this was going to be easy with autotools (autoconf etc), but I don't have any experience on using autoconf, so really can't say much on that. If you are saying cross compilation will require utils of other architecture, then I would say it's not tough. Debian and all it's fork supports multiaarch on the same machine, i.e you can install different aarch binaries on same machine.

I recently cross compiled gnu coreutils in my vm, it was very much easy. For example this is how I did(ignore the packages name, don't remember the exact name, just for ref):

Install the cross compiler toolchain

sudo apt-get install -y gcc-aarch64-linux-gnu

Install dependencies, note the aarch64 suffix, this will install aarch64 architecture package and will be installed at /usr/aarch64/bin (it will be some aarch64 directory, don't remember the exact location)

sudo apt-get install -y selinux:aarch64

configure suffix of cross compiler toolchain as host, the make command will be configured to use the cross compiler and it will configured to use the binaries, utils, libraries,dependencies from aarch64 location.

./configure --host=aarch64-linux-gnu

cross compilation begins

make

[gphalkes]

If Tilde were using autoconf, it would probably not be that difficult. However, Tilde does not use autoconf, but instead uses a custom configure script.

I may have been a bit unclear when I wrote that "a few utils are built that are run on the build system". What I meant was that during the build process, the build process itself tries to run some of the binaries that it just created. This doesn't work if the binary is for another architecture. However, looking up the instance of this I remember, I see that this has been replaced by a shell-script. So this isn't actually an issue.

I tried running the build using ./configure CC=arm-linux-gnueabihf-gcc-9 which theoretically should do the trick. However, this runs into the problem that libtool doesn't use the supplied compiler to run the linking, but instead uses the default gcc. That is of course for the build system, and not for the target system, so it fails. I didn't have time so far to figure out how to make this work, though some websites suggest that it needs a libtool specifically for the platform.

[fawazahmed0]

ok, I will try to cross compile when I get some free time and will post here, if I somehow succeed

[fawazahmed0]

seems like I am stuck on the same linking issue This is what I am getting:

[LD] src/tilde /usr/lib/gcc-cross/aarch64-linux-gnu/7/../../../../aarch64-linux-gnu/bin/ld: cannot find -ltranscript /usr/lib/gcc-cross/aarch64-linux-gnu/7/../../../../aarch64-linux-gnu/bin/ld: cannot find -ltranscript /usr/lib/gcc-cross/aarch64-linux-gnu/7/../../../../aarch64-linux-gnu/bin/ld: cannot find -lt3widget /usr/lib/gcc-cross/aarch64-linux-gnu/7/../../../../aarch64-linux-gnu/bin/ld: cannot find -lt3window /usr/lib/gcc-cross/aarch64-linux-gnu/7/../../../../aarch64-linux-gnu/bin/ld: cannot find -lt3config /usr/lib/gcc-cross/aarch64-linux-gnu/7/../../../../aarch64-linux-gnu/bin/ld: cannot find -lt3highlight /usr/lib/gcc-cross/aarch64-linux-gnu/7/../../../../aarch64-linux-gnu/bin/ld: cannot find -lt3config collect2: error: ld returned 1 exit status Makefile:83: recipe for target 'src/tilde' failed make: *** [src/tilde] Error 1

Even tried giving the libraries path in LDLIBS explicitly: ./configure CXX=aarch64-linux-gnu-g++ CXXFLAGS="-I/usr/aarch64-linux-gnu/include" LDLIBS="-L/usr/lib/aarch64-linux-gnu -ltranscript -lt3widget -lt3window -lt3config -lt3highlight -lt3config" && make LDFLAGS=-static Even that fails with the same error even though the arm64 libraries are already installed at that location. As you have told, this seems to be libtools issue, but really not sure about that: https://lists.debian.org/debian-devel/2011/02/msg00196.html

Not sure, if I could figure this out,because don't know much about libtool, might have to look into this in more depth in free time.

[fawazahmed0]

though some websites suggest that it needs a libtool specifically for the platform.

I assume you were referring to this: http://metastatic.org/text/libtool.html Tried everything what it says, but that doesn't seem to work

[fawazahmed0]

@gphalkes Do you know how to print debugging information while compilation and linking, I would like to see what libtool is trying to do (for example we have set -x in shell script or make -d , something similar to that)

Also do you know how to give libdir path to libtool during link phase, I tried printing help libtool --mode=link --help but I have no clue how to use that info Thanks

[gphalkes]

IIRC you need to run ./configure --verbose_compile for that. That will change the Makefile to print the commands that are being run. Or you can manually update the Makefile and clear out the values for the SILENT* variables.

[fawazahmed0]

@gphalkes Thanks, verbose actually helped to see what was going, I get the problem, as I am trying to build a static binary, ld command is trying to search for a static library which usually ends with .a, and my build system has .so shared libraries (for transcript, t3window etc) I have to compile library objects with -fPIC CFLAGS and then use ar command: ar rcs libtranscript.a *.o to make static library.

Do you have any plans to add support for static binary(and static library) building in ./t3shared/doall scripts etc?

[gphalkes]

I have no plans at this time, but if it can be added easily without cluttering up the build system, I might add it.

[gphalkes]

I had a little bit of time to play around with this. So far I've been successful in getting libt3config to at least compile with a cross compiler. To do this, I had to create a cross-libtool (I'm using arm-linux-gnueabihf for testing):

./configure --host=arm-linux-gnueabihf --program-prefix=arm-linux-gnueabihf- DESTDIR=/tmp make install

This installed the libtool script etc. in /tmp/usr/local. For testing this is OK, but obviously is not ideal.

Then I had to run the configure script for libt3config as follows:

LIBTOOL=/tmp/usr/local/bin/arm-linux-gnueabihf-libtool CC=arm-linux-gnueabihf-gcc ./configure --with-verbose-compile

For other parts of the Tilde compile, one needs to also add the CXX=arm-linux-gnueabihf-g++ to the compilation. Also, by replacing -shared by -static in the generated Makefile, I was able to create a static library.

I'll have to try to build the whole of Tilde this way to see if it is feasible, and then add the relevant prefixes in the configure script when using --host.

[fawazahmed0]

Thanks for looking into this, also just wanted add, you might have to use CXXFLAGS=-fPIC for libraries to be compiled as platform independent code, this is usually required when creating static libraries

[gphalkes]

I just wanted to point out that -fPIC is not necessary for static libraries, only for shared libraries. From the GCC documentation: "Generate position-independent code (PIC) suitable for use in a shared library". And this is precisely what using libtool is for: figuring out all the different options required for creating libraries, such as -fPIC :-)

One thing to note by the way, is that Tilde uses a plugin mechanism for supporting different clipboards. At this point only X11 is supported, but it does mean extra work for supporting static linking.

[fawazahmed0]

Thanks for clarifying me on that 👍