Closed lmangani closed 5 years ago
Interesting, I have not tried that but would like to. Are you cross-compiling from a different build machine? Can you tell me your steps so I can try to recreate? (I've not done a lot of cross-compiling so I many need some getting started pointers)
I'm building on an actual ARM64 platform, with regular tools - no issues compiling most other software so that's why I think its purely about missing definitions for aarch64
- a very similar thread in freeswitch at least suggested so.
yes, I saw that thread, and I think I have a similar issue in that config.sub is checked in, and it probably shouldn't be since it is generated by autogen.sh.
Here is what I tried on Debian 9 to cross compile to arm64, at least got past the configure step:
apt-get install crossbuild-essential-arm64
git clone https://github.com/davehorton/drachtio-server.git -b develop
cd drachtio-server/
git submodule update --init --recursive
rm config.sub
./autogen.sh
mkdir build && cd $_
../configure --host=aarch64-linux-gnu
make
still building so I can't tell if this is going to work, but can you try deleting config.sub and then building, starting with running ./autogen.sh
?
Next problem was that I needed an arm64 version of openssl and libcurl:
dpkg --add-architecture arm64
apt-get update
apt install libssl-dev:arm64 libcurl4-openssl-dev:arm64
..and I ran into trouble having concurrent libcurl installations for x86_64 and arm64, so I had to remove the x86_64:
apt-get remove libcurl4-openssl-dev
apt-get install apt-get install libcurl4-openssl-dev:arm64
Then everything compiled, but failed on the link step:
aarch64-linux-gnu-g++ -g -O2 -L/usr/local/lib -L../deps/prometheus-cpp/build/lib -L../deps/boost_1_69_0/stage/lib -o drachtio src/drachtio-main.o src/drachtio-controller.o src/drachtio-drachtio-config.o src/drachtio-client-controller.o src/drachtio-client.o src/drachtio-drachtio.o src/drachtio-sip-dialog.o src/drachtio-sip-dialog-controller.o src/drachtio-sip-proxy-controller.o src/drachtio-pending-request-controller.o src/drachtio-timer-queue.o src/drachtio-cdr.o src/drachtio-timer-queue-manager.o src/drachtio-sip-transports.o src/drachtio-request-handler.o src/drachtio-request-router.o src/drachtio-stats-collector.o ../deps/sofia-sip/libsofia-sip-ua/.libs/libsofia-sip-ua.a ../deps/jansson-2.10/src/.libs/libjansson.a -lcurl -lpthread -lssl -lcrypto -lz ../deps/boost_1_69_0/stage/lib/libboost_thread.a ../deps/boost_1_69_0/stage/lib/libboost_system.a ../deps/boost_1_69_0/stage/lib/libboost_log.a ../deps/boost_1_69_0/stage/lib/libboost_filesystem.a ../deps/prometheus-cpp/build/lib/libprometheus-cpp-pull.a ../deps/prometheus-cpp/build/lib/libprometheus-cpp-core.a -lpthread -lssl -lcrypto -lz -lrt
/usr/lib/gcc-cross/aarch64-linux-gnu/6/../../../../aarch64-linux-gnu/bin/ld: ../deps/sofia-sip/libsofia-sip-ua/.libs/libsofia-sip-ua.a(msg.o): Relocations in generic ELF (EM: 62)
/usr/lib/gcc-cross/aarch64-linux-gnu/6/../../../../aarch64-linux-gnu/bin/ld: ../deps/sofia-sip/libsofia-sip-ua/.libs/libsofia-sip-ua.a(msg.o): Relocations in generic ELF (EM: 62)
../deps/sofia-sip/libsofia-sip-ua/.libs/libsofia-sip-ua.a: error adding symbols: File in wrong format
collect2: error: ld returned 1 exit status
Makefile:582: recipe for target 'drachtio' failed
make[1]: *** [drachtio] Error 1
make[1]: Leaving directory '/usr/local/src/drachtio-server/build'
Makefile:446: recipe for target 'all' failed
make: *** [all] Error 2
I have a feeling that is because my Makefile.am includes a ./configure
in the sofia-sip submodule without specifying a --host
...so I had to go into the sofia-sip folder, delete config.sub there, re-run ./configure with the proper host, build that:
cd ../deps/sofia-sip/
make clean
rm config.sub
./bootstrap.sh
./configure --with-glib=no --host=aarch64-linux-gnu
make
Back to the drachtio build directory.....now sofia links (yay!)....but (sigh!) problem with libjansson:
root@debian-stretch-2vcpu-2gb-nyc3-01:/usr/local/src/drachtio-server/build# make
make all-am
make[1]: Entering directory '/usr/local/src/drachtio-server/build'
aarch64-linux-gnu-g++ -g -O2 -L/usr/local/lib -L../deps/prometheus-cpp/build/lib -L../deps/boost_1_69_0/stage/lib -o drachtio src/drachtio-main.o src/drachtio-controller.o src/drachtio-drachtio-config.o src/drachtio-client-controller.o src/drachtio-client.o src/drachtio-drachtio.o src/drachtio-sip-dialog.o src/drachtio-sip-dialog-controller.o src/drachtio-sip-proxy-controller.o src/drachtio-pending-request-controller.o src/drachtio-timer-queue.o src/drachtio-cdr.o src/drachtio-timer-queue-manager.o src/drachtio-sip-transports.o src/drachtio-request-handler.o src/drachtio-request-router.o src/drachtio-stats-collector.o ../deps/sofia-sip/libsofia-sip-ua/.libs/libsofia-sip-ua.a ../deps/jansson-2.10/src/.libs/libjansson.a -lcurl -lpthread -lssl -lcrypto -lz ../deps/boost_1_69_0/stage/lib/libboost_thread.a ../deps/boost_1_69_0/stage/lib/libboost_system.a ../deps/boost_1_69_0/stage/lib/libboost_log.a ../deps/boost_1_69_0/stage/lib/libboost_filesystem.a ../deps/prometheus-cpp/build/lib/libprometheus-cpp-pull.a ../deps/prometheus-cpp/build/lib/libprometheus-cpp-core.a -lpthread -lssl -lcrypto -lz -lrt
/usr/lib/gcc-cross/aarch64-linux-gnu/6/../../../../aarch64-linux-gnu/bin/ld: ../deps/jansson-2.10/src/.libs/libjansson.a(load.o): Relocations in generic ELF (EM: 62)
/usr/lib/gcc-cross/aarch64-linux-gnu/6/../../../../aarch64-linux-gnu/bin/ld: ../deps/jansson-2.10/src/.libs/libjansson.a(load.o): Relocations in generic ELF (EM: 62)
/usr/lib/gcc-cross/aarch64-linux-gnu/6/../../../../aarch64-linux-gnu/bin/ld: ../deps/jansson-2.10/src/.libs/libjansson.a(load.o): Relocations in generic ELF (EM: 62)
/usr/lib/gcc-cross/aarch64-linux-gnu/6/../../../../aarch64-linux-gnu/bin/ld: ../deps/jansson-2.10/src/.libs/libjansson.a(load.o): Relocations in generic ELF (EM: 62)
/usr/lib/gcc-cross/aarch64-linux-gnu/6/../../../../aarch64-linux-gnu/bin/ld: ../deps/jansson-2.10/src/.libs/libjansson.a(load.o): Relocations in generic ELF (EM: 62)
/usr/lib/gcc-cross/aarch64-linux-gnu/6/../../../../aarch64-linux-gnu/bin/ld: ../deps/jansson-2.10/src/.libs/libjansson.a(load.o): Relocations in generic ELF (EM: 62)
../deps/jansson-2.10/src/.libs/libjansson.a: error adding symbols: File in wrong format
collect2: error: ld returned 1 exit status
Makefile:582: recipe for target 'drachtio' failed
make[1]: *** [drachtio] Error 1
make[1]: Leaving directory '/usr/local/src/drachtio-server/build'
Makefile:446: recipe for target 'all' failed
make: *** [all] Error 2
...so, same deal to fix jansson:
cd ../deps/jansson-2.10/
make clean
rm config.sub
autoreconf -fi
./configure --host=aarch64-linux-gnu
make
and, that worked...but of course, now there is a problem with our one other thus-far unmentioned dependency, which is the boost library
root@debian-stretch-2vcpu-2gb-nyc3-01:/usr/local/src/drachtio-server/build# make
make all-am
make[1]: Entering directory '/usr/local/src/drachtio-server/build'
aarch64-linux-gnu-g++ -DHAVE_CONFIG_H -I. -I.. -I../deps/boost_1_69_0 -I../deps/sofia-sip/libsofia-sip-ua/su -I../deps/sofia-sip/libsofia-sip-ua/nta -I../deps/sofia-sip/libsofia-sip-ua/sip -I../deps/sofia-sip/libsofia-sip-ua/msg -I../deps/sofia-sip/libsofia-sip-ua/url -I../deps/sofia-sip/libsofia-sip-ua/tport -I../deps/sofia-sip/libsofia-sip-ua/bnf -I../deps/jansson-2.10/src -I../deps/prometheus-cpp/build/include -I/usr/local/include -D_REENTRANT -DDRACHTIO_VERSION=\"v0.8.1-rc1\" -Wno-error=deprecated-declarations -DBOOST_ALLOW_DEPRECATED_HEADERS -g -O2 -MT src/drachtio-controller.o -MD -MP -MF src/.deps/drachtio-controller.Tpo -c -o src/drachtio-controller.o `test -f 'src/controller.cpp' || echo '../'`src/controller.cpp
mv -f src/.deps/drachtio-controller.Tpo src/.deps/drachtio-controller.Po
aarch64-linux-gnu-g++ -DHAVE_CONFIG_H -I. -I.. -I../deps/boost_1_69_0 -I../deps/sofia-sip/libsofia-sip-ua/su -I../deps/sofia-sip/libsofia-sip-ua/nta -I../deps/sofia-sip/libsofia-sip-ua/sip -I../deps/sofia-sip/libsofia-sip-ua/msg -I../deps/sofia-sip/libsofia-sip-ua/url -I../deps/sofia-sip/libsofia-sip-ua/tport -I../deps/sofia-sip/libsofia-sip-ua/bnf -I../deps/jansson-2.10/src -I../deps/prometheus-cpp/build/include -I/usr/local/include -D_REENTRANT -DDRACHTIO_VERSION=\"v0.8.1-rc1\" -Wno-error=deprecated-declarations -DBOOST_ALLOW_DEPRECATED_HEADERS -g -O2 -MT src/drachtio-request-handler.o -MD -MP -MF src/.deps/drachtio-request-handler.Tpo -c -o src/drachtio-request-handler.o `test -f 'src/request-handler.cpp' || echo '../'`src/request-handler.cpp
mv -f src/.deps/drachtio-request-handler.Tpo src/.deps/drachtio-request-handler.Po
aarch64-linux-gnu-g++ -g -O2 -L/usr/local/lib -L../deps/prometheus-cpp/build/lib -L../deps/boost_1_69_0/stage/lib -o drachtio src/drachtio-main.o src/drachtio-controller.o src/drachtio-drachtio-config.o src/drachtio-client-controller.o src/drachtio-client.o src/drachtio-drachtio.o src/drachtio-sip-dialog.o src/drachtio-sip-dialog-controller.o src/drachtio-sip-proxy-controller.o src/drachtio-pending-request-controller.o src/drachtio-timer-queue.o src/drachtio-cdr.o src/drachtio-timer-queue-manager.o src/drachtio-sip-transports.o src/drachtio-request-handler.o src/drachtio-request-router.o src/drachtio-stats-collector.o ../deps/sofia-sip/libsofia-sip-ua/.libs/libsofia-sip-ua.a ../deps/jansson-2.10/src/.libs/libjansson.a -lcurl -lpthread -lssl -lcrypto -lz ../deps/boost_1_69_0/stage/lib/libboost_thread.a ../deps/boost_1_69_0/stage/lib/libboost_system.a ../deps/boost_1_69_0/stage/lib/libboost_log.a ../deps/boost_1_69_0/stage/lib/libboost_filesystem.a ../deps/prometheus-cpp/build/lib/libprometheus-cpp-pull.a ../deps/prometheus-cpp/build/lib/libprometheus-cpp-core.a -lpthread -lssl -lcrypto -lz -lrt
/usr/lib/gcc-cross/aarch64-linux-gnu/6/../../../../aarch64-linux-gnu/bin/ld: cannot find -lz
/usr/lib/gcc-cross/aarch64-linux-gnu/6/../../../../aarch64-linux-gnu/bin/ld: ../deps/boost_1_69_0/stage/lib/libboost_thread.a(thread.o): Relocations in generic ELF (EM: 62)
/
plus we need libz, so:
apt-get install libz-dev:arm64
It's occuring to me that most of this will not hit you since you are building on an ARM machine -- most of what I am working through has to do with cross-compiling. I think deleting the config.sub files in the root project folder and deps/sofia-sip may be all that you need to do...
Nonetheless, for posterity sake in detailing how to do a cross-compile, onwards I go!
So, to cross-compile boost for arm64 I had to:
Here were my commands:
cd ../deps/boost_1_69_0/
./b2 --clean
rm bin.v2/project-cache.jam
./bootstrap.sh
view project-config.jam
and here, I then changed this lines:
using gcc ;
to this:
using gcc : arm :aarch64-linux-gnu-g++ ;
and, finally, build boost
./b2 architecture=arm address-model=64 link=static variant=release stage
Now, did that get me to the promised land? Not quite:
root@debian-stretch-2vcpu-2gb-nyc3-01:/usr/local/src/drachtio-server/build# make
make all-am
make[1]: Entering directory '/usr/local/src/drachtio-server/build'
aarch64-linux-gnu-g++ -g -O2 -L/usr/local/lib -L../deps/prometheus-cpp/build/lib -L../deps/boost_1_69_0/stage/lib -o drachtio src/drachtio-main.o src/drachtio-controller.o src/drachtio-drachtio-config.o src/drachtio-client-controller.o src/drachtio-client.o src/drachtio-drachtio.o src/drachtio-sip-dialog.o src/drachtio-sip-dialog-controller.o src/drachtio-sip-proxy-controller.o src/drachtio-pending-request-controller.o src/drachtio-timer-queue.o src/drachtio-cdr.o src/drachtio-timer-queue-manager.o src/drachtio-sip-transports.o src/drachtio-request-handler.o src/drachtio-request-router.o src/drachtio-stats-collector.o ../deps/sofia-sip/libsofia-sip-ua/.libs/libsofia-sip-ua.a ../deps/jansson-2.10/src/.libs/libjansson.a -lcurl -lpthread -lssl -lcrypto -lz ../deps/boost_1_69_0/stage/lib/libboost_thread.a ../deps/boost_1_69_0/stage/lib/libboost_system.a ../deps/boost_1_69_0/stage/lib/libboost_log.a ../deps/boost_1_69_0/stage/lib/libboost_filesystem.a ../deps/prometheus-cpp/build/lib/libprometheus-cpp-pull.a ../deps/prometheus-cpp/build/lib/libprometheus-cpp-core.a -lpthread -lssl -lcrypto -lz -lrt
/usr/lib/gcc-cross/aarch64-linux-gnu/6/../../../../aarch64-linux-gnu/bin/ld: ../deps/prometheus-cpp/build/lib/libprometheus-cpp-pull.a(exposer.cc.o): Relocations in generic ELF (EM: 62)
Ah yes. I'd forgotten about the prometheus library I just added. Which is a CMAKE build, so that should be fun trying to cross-compile...
Wow @davehorton thank you so much, that's plain amazing! I'm going to validate all of this next and confirm results + a nice arm64 package of course ;)
at the moment I am kind of stumped on how to cross-compile the prometheus library, due to my lack of familiarity with cmake.
I created a toolchain file called arm64.cmake that looks like this:
SET(CMAKE_SYSTEM_NAME Linux)
SET(CMAKE_C_COMPILER aarch64-linux-gnu-gcc)
SET(CMAKE_CXX_COMPILER aarch64-linux-gnu-g++)
SET(CMAKE_FIND_ROOT_PATH /usr/lib/aarch64-linux-gnu/pkgconfig)
SET(CMAKE_FIND_ROOT_PATH_MODE_PROGRAM NEVER)
SET(CMAKE_FIND_ROOT_PATH_MODE_LIBRARY ONLY)
SET(CMAKE_FIND_ROOT_PATH_MDOE_INCLUDE ONLY)
Build fails however because it cant find zlib or libcurl (even though *.pc files are in the pkgconfig directory I specified above)
root@debian-stretch-2vcpu-2gb-nyc3-01:/usr/local/src/drachtio-server/deps/prometheus-cpp/build# cmake .. -DCMAKE_TOOLCHAIN_FILE=../arm64.cmake -DBUILD_SHARED_LIBS=OFF -DCMAKE_INSTALL_LIBDIR=lib -DCMAKE_INSTALL_PREFIX=`pwd`
-- The C compiler identification is GNU 6.3.0
-- The CXX compiler identification is GNU 6.3.0
-- Check for working C compiler: /usr/bin/aarch64-linux-gnu-gcc
-- Check for working C compiler: /usr/bin/aarch64-linux-gnu-gcc -- works
-- Detecting C compiler ABI info
-- Detecting C compiler ABI info - done
-- Detecting C compile features
-- Detecting C compile features - done
-- Check for working CXX compiler: /usr/bin/aarch64-linux-gnu-g++
-- Check for working CXX compiler: /usr/bin/aarch64-linux-gnu-g++ -- works
-- Detecting CXX compiler ABI info
-- Detecting CXX compiler ABI info - done
-- Detecting CXX compile features
-- Detecting CXX compile features - done
-- Looking for pthread.h
-- Looking for pthread.h - found
-- Looking for pthread_create
-- Looking for pthread_create - not found
-- Looking for pthread_create in pthreads
-- Looking for pthread_create in pthreads - not found
-- Looking for pthread_create in pthread
-- Looking for pthread_create in pthread - found
-- Found Threads: TRUE
-- Could NOT find GoogleBenchmark (missing: GoogleBenchmark_LIBRARY GoogleBenchmark_INCLUDE_DIR)
CMake Error at /usr/share/cmake-3.7/Modules/FindPackageHandleStandardArgs.cmake:138 (message):
Could NOT find ZLIB (missing: ZLIB_LIBRARY) (found version "1.2.8")
Anyways, let me know how the native arm64 build comes out..
Just in case this could ease the pain, this is how I'm building my packages in Docker w/o cross-compiling inside qemu aarch64 emulation: https://github.com/QXIP/EXA8/tree/master/builders/opensips
Obviously comes with a different set of limitations (for GO mostly) but hopefully unimpacting here
Thanks I will take a look.
I just pushed a commit on the develop branch that removes the config.sub files so they can be dynamically generated by the build process. This should make things build natively on arm64? Have you had any luck?
I forgot I had an old raspberry pi lying around, still in the box, so I took that out and fired it up. With the latest on the develop branch (including commit referenced above), ./configure
works fine and it is now building. I had only to install autoconf and libcurl4-openssl-dev to the stock os config.
This is arm32, but I think arm64 would work as well.
It's still building so I will update with results once done
ACK - Its passing the configure but now failing with libboost
which i'm trying to sort out - its forcing -m64
flags for me
ok....boost did build for me on the arm32 hardware. boost detected both that the processor was arm and the address model 32 as part of the standard config
I think that's the culprit, on my distro (ubuntu xenial on a Cavium SOC) the aarch64 architecture or host is not detected and this spirals downwards. I wonder if I can force this anywhere....
ok, for posterity, everthing built and seems run fine on arm32 with no changes. I did:
stock raspberry pi install, then
apt-get install autoconf libcurl4-openssl-dev libtool-bin libssl-dev cmake
then following the standard build instructions work.
I've ordered a newer 64-bit raspberry pi so I can test with that as well
Thanks Dave! Here's my current builder displaying the condition if you want to try it in emulation mode: https://github.com/QXIP/EXA8/tree/dev/builders/drachtio
ok, thanks, will try to take a run at that later
OK, I think I have a grip on this now. I need to remove a few more cache files from my codebase, then it will build quite simply. I'll give you a PR for your build.sh once I verify it
OK, the latest on the develop branch builds on arm64 with the standard build instructions. I also submitted a PR for your build script. I'll wait until you verify you can build now before closing this issue.
any luck building with the latest from develop now on arm64?
machine says YES - or at least there's something in the package I found after a looooong compile :)
-rw-r--r-- 1 root root 3753156 Apr 18 01:59 drachtio_0.8-1.xenial.arm64.deb
total 15424
drwxr-xr-x 2 root root 4096 Apr 18 01:59 ./
drwxr-xr-x 4 root root 4096 Apr 18 01:59 ../
-rwxr-xr-x 1 root root 10555416 Apr 18 01:55 drachtio*
-rwxr-xr-x 1 root root 363104 Apr 18 01:55 parser*
-rwxr-xr-x 1 root root 53864 Apr 18 01:55 ssltest*
-rwxr-xr-x 1 root root 1325944 Apr 18 01:55 test_asio_curl*
-rwxr-xr-x 1 root root 2523112 Apr 18 01:55 test_https*
-rwxr-xr-x 1 root root 141632 Apr 18 01:55 timer*
-rwxr-xr-x 1 root root 810120 Apr 18 01:55 uri_test*
I'll test this and make sure the fine details are smooth for general use next. Do I need the other files? I think I noticed somewhere in your process a few being removed.
Thanks @davehorton you're a superstar!
no, you can delete all of those other binaries, they are random test tools, not needed for drachtio
also, not sure how you are intending to run it on your hardware, but I wanted to note that you can use either command line parameters or environment variables for most of the settings if it is a pain to have a config file
thanks you so much for helping! I think I owe you some time whenever you'd like ;)
Closing issue +1
Hi i am getting this error please help me
bin/bash: line 2: 20096 Segmentation fault (core dumped) cmake .. -DBUILD_SHARED_LIBS=OFF -DCMAKE_INSTALL_LIBDIR=lib -DCMAKE_INSTALL_PREFIX=pwd
Makefile:1398: recipe for target '../deps/prometheus-cpp/build/lib/libprometheus-cpp-core.a' failed
make: *** [../deps/prometheus-cpp/build/lib/libprometheus-cpp-core.a] Error 139
the fact that core dumped makes me think you may not have enough RAM on the server to complete the build. What are the specs of the machine you are running on?
By using VM running in Linux and I allocated size as 30 GB
what linux distro are you using? are you also on arm64 as the original issue poster was? You are building the 'develop' branch, correct?
I'm using linuxmint 64bit...no not arm64..yes you are right I'm trying to build the drachtio-server.when I gave "make" command, the above error is coming
OK can you show all the commands that you ran up to that point in order to build? I will try to create on linuxmint. By the way, do you use ansible? If so I would ask you to try installing using my ansible role.
git clone --depth=50 --branch=develop git://github.com/davehorton/drachtio-server.git && cd drachtio-server git submodule update --init --recursive ./autogen.sh mkdir build && cd $_ ../configure CPPFLAGS='-DNDEBUG' make sudo make install
I am not using ansible but is this different from Sofia SIP stack
I can successfully build on ARM64 with the following
apt-get install -y build-essential git libssl-dev libtool libtool-bin autoconf \
automake zlib1g-dev cmake libcurl4-openssl-dev
cd /usr/src
git clone --depth=50 --branch=develop git://github.com/davehorton/drachtio-server.git && cd drachtio-server
git submodule update --init --recursive
./autogen.sh
mkdir build && cd $_
../configure CPPFLAGS='-DNDEBUG' CXXFLAGS='-O0'
make
make install exec_prefix=${TMP_DIR} prefix=${TMP_DIR}
@susmidha-hub when you say "I allocated size as 30 GB" it sounds to me like you allocated a 30G disk for the VM. How much RAM?
When trying to build for aarch64/arm64 platforms the procedure fails with:
I've tried common patterns but the fall unrecognized for arm64, potentially caused by outdated
config.sub
files from pre-arm eras. Is this something anyone already encountered/solved perhaps?