search

roc-streaming / roc-toolkit

Real-time audio streaming over the network.
https://roc-streaming.org
Mozilla Public License 2.0
1.06k stars 213 forks source link

Android/Termux build #222

Closed S-trace closed 5 years ago

S-trace commented 5 years ago

Hello.

I'd like to use roc pulseaudio module on PC and Android to stream audio from PC to Android (now I'm using module-tunnel-sink-new on the PC and module-native-protocol-tcp on the Android - it works, but glitching sometimes).

Trying to build roc using Termux packages builder from https://github.com/termux/unstable-packages but build fails during configure.

Could You please help me with porting roc to the Android/Termux? Build scripts for Termux packages (scons, libopenfec and roc): packages.zip

Thank You.

gavv commented 5 years ago

Great! I'll take a look.

I'm not familiar with termux. How to build this? I guess I should clone termux/unstable-packages, add the contents of this archive to the packages/ directory and run build-package.sh?

It's possible to build termux packages on PC, right? In this case, will be they cross-compiled or built for the local architecture (x86_64-pc-linux-gnu)?

UPD: Ah, it seems that I can choose the target architecture.

S-trace commented 5 years ago

Please clone termux/unstable-packages, add the contents of this archive to the packages/ directory, run start-builder.sh (it will setup docker container (~1.5GB) with build evnironment - you will need docker on the build machine to do this) and then (inside build container) run ./build-package.sh -i roc - it will download prebuild libraries for existing deps, build new packages (scons, libopenfec) and will try to build roc. After first build attempt you can run ./build-package.sh -s roc to skip deps check - this speeds up rebuilding a lot.

Termux packages are cross-compiled, building on target device is not supported.

gavv commented 5 years ago

Thanks. I've reproduced the problem. I'll try to fix it.

gavv commented 5 years ago

First of all, thanks for your work, it was a lot easier for me to start from a ready-to-use environment (termux, your scripts, and your instructions). This step is usually the most time-consuming.

I've pushed a series commits to the develop branch that fix various issues in scons:

I've also added a new android platform and adjusted the build process for it. It differs from linux by some compiler options and the standard library version (glibc vs bionic).

With these changes, I was able to build your termux package, but so far without PulseAudio modules.

Below you can find the updated build scripts.

libopenfec/build.sh

Changes:

TERMUX_PKG_HOMEPAGE=http://openfec.org
TERMUX_PKG_DESCRIPTION="Application-Level Forward Erasure Correction implementation library"
TERMUX_PKG_LICENSE="CeCCIL-C"
TERMUX_PKG_VERSION=1.4.2.3
TERMUX_PKG_SHA256=5cc47e403a62afcb1f0423390d6d1a370f0bf969aa01b79fdfd1ad4852b59aff
TERMUX_PKG_SRCURL=https://github.com/roc-project/openfec/archive/v$TERMUX_PKG_VERSION.tar.gz

# Cmake definition
TERMUX_PKG_EXTRA_CONFIGURE_ARGS="
-DTERMUX:STRING=TERMUX
"

termux_step_install_license() {
    install -Dm644 "$TERMUX_TOPDIR/$TERMUX_PKG_NAME/src/LICENCE_CeCILL-C_V1-en.txt" "$TERMUX_PREFIX/share/doc/$TERMUX_PKG_NAME/LICENCE_CeCILL-C_V1-en.txt"
    install -Dm644 "$TERMUX_TOPDIR/$TERMUX_PKG_NAME/src/LICENCE_CeCILL-C_V1-en.txt" "$TERMUX_PREFIX/share/doc/$TERMUX_PKG_NAME/Licence_CeCILL_V2-en.txt"
}

termux_step_make_install() {
    (
        cd  "$TERMUX_TOPDIR/$TERMUX_PKG_NAME/src/src"
        find -type f -name '*.h' | while read f; do
            install -Dm644 "$f" "$TERMUX_PREFIX/include/openfec/$f"
        done
    )
    install -Dm644 "$TERMUX_TOPDIR/$TERMUX_PKG_NAME/src/bin/Release/libopenfec.so" "$TERMUX_PREFIX/lib/libopenfec.so"p
}

roc/build.sh

Changes:

TERMUX_PKG_HOMEPAGE=https://roc-project.github.io
TERMUX_PKG_DESCRIPTION="Roc real-time streaming over the network"
TERMUX_PKG_LICENSE="LGPL-2.0, MPL-2.0"
TERMUX_PKG_VERSION=0.1.0
TERMUX_PKG_SHA256=20cddc35127c2b6e93bbcaf9d577d6d3cf890f7ab374716180b2747ed4b0a51b
TERMUX_PKG_SRCURL=https://github.com/roc-project/roc/archive/ecd048e24f5dfc9756afdb5d72a5219abb2cd850.tar.gz
TERMUX_PKG_BUILD_DEPENDS="python, pkg-config, scons"
TERMUX_PKG_DEPENDS="libuv, libpulseaudio, libopenfec"

termux_step_install_license() {
    install -Dm644 "$TERMUX_TOPDIR/$TERMUX_PKG_NAME/src/LICENSE" "$TERMUX_PREFIX/share/doc/$TERMUX_PKG_NAME/LICENSE_MPL-2.0.txt"
    install -Dm644 "$TERMUX_TOPDIR/$TERMUX_PKG_NAME/src/src/pulse/LICENSE" "$TERMUX_PREFIX/share/doc/$TERMUX_PKG_NAME/LICENSE_LGPL-2.0.txt"
}

run_scons() {
        cp "$TERMUX_TOPDIR/_cache/19b-aarch64-24-v5/sysroot/usr/lib/aarch64-linux-android/24/libdl.so" "$TERMUX_PREFIX/lib"
        scons --prefix="$TERMUX_PREFIX" --host="$CCTERMUX_HOST_PLATFORM" --with-openfec-includes="/data/data/com.termux/files/usr/include/openfec" --compiler=clang --disable-tools --disable-tests --disable-examples --disable-doc --disable-sox "$@"
}

termux_step_make_install() {
   (
        cd "$TERMUX_TOPDIR/$TERMUX_PKG_NAME/src/" || return 1
        run_scons install
    )
}

termux_step_make() {
    (
        cd "$TERMUX_TOPDIR/$TERMUX_PKG_NAME/src/" || return 1
        run_scons
    )
}
gavv commented 5 years ago

What's next:

gavv commented 5 years ago

Also, would you like to contribute your OpenFEC patch to https://github.com/roc-project/openfec? It would allow us in future to build Roc for android without termux. I think it is unlikely that the OpenFEC upstream will accept any patches in the nearest future (they told they have no time for this now), so for now we're stuck on our fork.

S-trace commented 5 years ago

If Roc will be built against internal pulseaudio - wouldn't this break the,module,if private headers of internal and system pulseaudios will be different?

I'm think we can just pass $TERMUX_TOPDIR/libpulseaudio/src/ as path to the system pulseaudio sources to make sure roc pulseaudio module will be built against system pulseaudio private headers.

Could You please take a look on https://github.com/termux/unstable-packages/commits/master commits on Jun 15, 2019 - libopenfec and roc (with pulseaudio module) now is in the Termux repository. But pulseaudio module fails to load: 

E: [pulseaudio] ltdl-bind-now.c: Failed to open module /data/data/com.termux/files/usr/lib/pulse-12.2/modules/module-roc-sink-input.so: dlopen failed: cannot find "libroc.so" from verneed[1] in DT_NEEDED list for "/data/data/com.termux/files/usr/lib/pulse-12.2/modules/module-roc-sink-input.so"
E: [pulseaudio] module.c: Failed to open module "module-roc-sink-input".

/data/data/com.termux/files/usr/lib/libroc.so exists:

$ file /data/data/com.termux/files/usr/lib/libroc.so
/data/data/com.termux/files/usr/lib/libroc.so: ELF 64-bit LSB shared object, ARM aarch64, version 1 (SYSV), dynamically linked, not stripped

UPD: Tried to use system pulseaudio:

run_scons() {
        cp -r "$TERMUX_TOPDIR/libpulseaudio/build"/* "$TERMUX_TOPDIR/libpulseaudio/src"
        scons --prefix="$TERMUX_PREFIX" --host="$CCTERMUX_HOST_PLATFORM" --with-openfec-includes="/data/data/com.termux/files/usr/include/openfec" --compiler=clang --disable-tools --disable-tests --disable-examples --disable-doc --disable-sox --enable-pulseaudio-modules --with-pulseaudio=$TERMUX_TOPDIR/libpulseaudio/src "$@"
}

after this rebuild pulseaudio and roc packages:

rm /data/data/.built-packages/libpulseaudio -f ; ./build-package.sh -s libpulseaudio
rm /data/data/.built-packages/roc -f ; ./build-package.sh -s roc

Pulseaudio modules were built, but pulseaudio can't load them with same error: Failed to open module /data/data/com.termux/files/usr/lib/pulse-12.2/modules/module-roc-sink-input.so: dlopen failed: cannot find "libroc.so" from verneed[1] in DT_NEEDED list for "/data/data/com.termux/files/usr/lib/pulse-12.2/modules/module-roc-sink-input.so"

UPD2: Looks like we need something like --with-pulseaudio-includes=$TERMUX_TOPDIR/libpulseaudio/src scons parameter to get rid of cp -r "$TERMUX_TOPDIR/libpulseaudio/build"/* "$TERMUX_TOPDIR/libpulseaudio/src" dirty hack

gavv commented 5 years ago

If Roc will be built against internal pulseaudio - wouldn't this break the,module,if private headers of internal and system pulseaudios will be different?

It will. Versions should be exactly the same.

I'm think we can just pass $TERMUX_TOPDIR/libpulseaudio/src/ as path to the system pulseaudio sources to make sure roc pulseaudio module will be built against system pulseaudio private headers.

Should work, but the drawback is that it would not be possible to install binary package for PulseAudio. You'll have to explicitly build PulseAudio before building Roc.

Could You please take a look on https://github.com/termux/unstable-packages/commits/master commits on Jun 15, 2019 - libopenfec and roc (with pulseaudio module) now is in the Termux repository.

Wow. That SConstruct patch is probably not necessary anymore for fresh develop branch...

Looks like we need something like --with-pulseaudio-includes=$TERMUX_TOPDIR/libpulseaudio/src scons parameter to get rid of cp -r "$TERMUX_TOPDIR/libpulseaudio/build"/* "$TERMUX_TOPDIR/libpulseaudio/src" dirty hack

I've tried this approach and failed because $TERMUX_TOPDIR/libpulseaudio/src and $TERMUX_TOPDIR/libpulseaudio/build are both include directories (the first one contains source files and the second contains config.h generated by autoconf).

Instead, I've added --with-pulseaudio-build-dir option that can be used in case of out-of-source builds like this. Pushed to the develop branch.

Here is how you can use it:

scons --prefix="$TERMUX_PREFIX" \
    --host="$CCTERMUX_HOST_PLATFORM" \
    --with-openfec-includes="/home/builder/.termux-build/libopenfec/src/src" \
    --compiler=clang \
    --disable-tests --disable-examples --disable-doc --disable-sox \
    --enable-pulseaudio-modules \
    --with-pulseaudio=/home/builder/.termux-build/libpulseaudio/src \
    --with-pulseaudio-build-dir=/home/builder/.termux-build/libpulseaudio/build \
    "$@"

Works for me.

Pulseaudio modules were built, but pulseaudio can't load them with same error:

Could you check whether module-roc-sink-input.so has RUNPATH set?

In my case it has:

/opt/termux/android-sdk/ndk-bundle/toolchains/llvm/prebuilt/linux-x86_64/bin/aarch64-linux-android-readelf -d /data/data/com.termux/files/usr/lib/pulse-12.2/modules/module-roc-sink-input.so | grep PATH
 0x000000000000001d (RUNPATH)            Library runpath: [/data/data/com.termux/files/usr/lib]

If there is no RUNPATH, it's probably a build issue. If there is, the problem is probably that dlopen() invoked from PulseAudio daemon just ignores paths set for PulseAudio modules...

In this case the solution would be to run PulseAudio with LD_LIBRARY_PATH:

export LD_LIBRARY_PATH="$LD_LIBRARY_PATH:/data/data/com.termux/files/usr/lib"
pulseaudio --start

Could you please try this?

S-trace commented 5 years ago

Could you please try this? Tried this - roc module failed to load as before, and module-sles-sink (android's sles sink) also failed to load:

E: [pulseaudio] ltdl-bind-now.c: Failed to open module /data/data/com.termux/files/usr/lib/pulse-12.2/modules/module-sles-sink.so: dlopen failed: cannot locate symbol "XzUnpacker_Construct" referenced by "/system/lib64/libunwind.so"...

Could you check whether module-roc-sink-input.so has RUNPATH set? It is, both in module-roc-sink-input.so and libroc.so: 


$ readelf -d /data/data/com.termux/files/usr/lib/pulse-12.2/modules/module-roc-sink-input.so

Dynamic section at offset 0xfd70 contains 32 entries: Tag Type Name/Value 0x0000000000000003 (PLTGOT) 0x1ffe8 0x0000000000000002 (PLTRELSZ) 1368 (bytes) 0x0000000000000017 (JMPREL) 0x1440 0x0000000000000014 (PLTREL) RELA 0x0000000000000007 (RELA) 0x12c0 0x0000000000000008 (RELASZ) 384 (bytes) 0x0000000000000009 (RELAENT) 24 (bytes) 0x000000006ffffff9 (RELACOUNT) 12 0x0000000000000006 (SYMTAB) 0x298 0x000000000000000b (SYMENT) 24 (bytes) 0x0000000000000005 (STRTAB) 0x988 0x000000000000000a (STRSZ) 1542 (bytes) 0x0000000000000004 (HASH) 0xf90 0x0000000000000001 (NEEDED) Shared library: [libuv.so] 0x0000000000000001 (NEEDED) Shared library: [libdl.so] 0x0000000000000001 (NEEDED) Shared library: [libopenfec.so] 0x0000000000000001 (NEEDED) Shared library: [libltdl.so] 0x0000000000000001 (NEEDED) Shared library: [libpulsecore-12.2.so] 0x0000000000000001 (NEEDED) Shared library: [libpulsecommon-12.2.so] 0x0000000000000001 (NEEDED) Shared library: [libroc.so] 0x0000000000000001 (NEEDED) Shared library: [libc++_shared.so] 0x0000000000000001 (NEEDED) Shared library: [libm.so] 0x0000000000000001 (NEEDED) Shared library: [libc.so] 0x000000000000001a (FINI_ARRAY) 0x1fd10 0x000000000000001c (FINI_ARRAYSZ) 16 (bytes) 0x000000000000001d (RUNPATH) Library runpath: [/data/data/com.termux/files/usr/lib] 0x000000006ffffff0 (VERSYM) 0x11cc 0x000000006ffffffc (VERDEF) 0x1260 0x000000006ffffffd (VERDEFNUM) 1 0x000000006ffffffe (VERNEED) 0x127c 0x000000006fffffff (VERNEEDNUM) 2 0x0000000000000000 (NULL) 0x0 $ readelf -d /data/data/com.termux/files/usr/lib/libroc.so

Dynamic section at offset 0x2fd78 contains 28 entries: Tag Type Name/Value 0x0000000000000003 (PLTGOT) 0x3ffe8 0x0000000000000002 (PLTRELSZ) 1584 (bytes) 0x0000000000000017 (JMPREL) 0x2b70 0x0000000000000014 (PLTREL) RELA 0x0000000000000007 (RELA) 0x13b8 0x0000000000000008 (RELASZ) 6072 (bytes) 0x0000000000000009 (RELAENT) 24 (bytes) 0x000000006ffffff9 (RELACOUNT) 216 0x0000000000000006 (SYMTAB) 0x260 0x000000000000000b (SYMENT) 24 (bytes) 0x0000000000000005 (STRTAB) 0xaa0 0x000000000000000a (STRSZ) 1393 (bytes) 0x0000000000000004 (HASH) 0x1018 0x0000000000000001 (NEEDED) Shared library: [libuv.so] 0x0000000000000001 (NEEDED) Shared library: [libdl.so] 0x0000000000000001 (NEEDED) Shared library: [libopenfec.so] 0x0000000000000001 (NEEDED) Shared library: [libc++_shared.so] 0x0000000000000001 (NEEDED) Shared library: [libm.so] 0x0000000000000001 (NEEDED) Shared library: [libc.so] 0x000000000000001a (FINI_ARRAY) 0x3f2d0 0x000000000000001c (FINI_ARRAYSZ) 16 (bytes) 0x000000000000001d (RUNPATH) Library runpath: [/data/data/com.termux/files/usr/lib] 0x000000006ffffff0 (VERSYM) 0x128c 0x000000006ffffffc (VERDEF) 0x133c 0x000000006ffffffd (VERDEFNUM) 2 0x000000006ffffffe (VERNEED) 0x1374 0x000000006fffffff (VERNEEDNUM) 2 0x0000000000000000 (NULL) 0x0


Tried to trace dlopen calls from pulseaudio using gdb:

gdb pulseaudio break dlopen commands print (char*)$x0 print (int)$x1 continue end run

And got this:

... Thread 1 "pulseaudio" hit Breakpoint 1, 0x0000007fb794cf04 in dlopen () from /system/lib64/libdl.so $35 = 0x7fb6e4a8c0 "/data/data/com.termux/files/usr/lib/pulse-12.2/modules/module-sles-sink.so" $36 = 2 ... Thread 1 "pulseaudio" hit Breakpoint 1, 0x0000007fb794cf04 in dlopen () from /system/lib64/libdl.so $53 = 0x7fb6e4b7c0 "/data/data/com.termux/files/usr/lib/pulse-12.2/modules/module-roc-sink-input.so" $54 = 2 E: [pulseaudio] ltdl-bind-now.c: Failed to open module /data/data/com.termux/files/usr/lib/pulse-12.2/modules/module-roc-sink-input.so: dlopen failed: cannot find "libroc.so" from verneed[1] in DT_NEEDED list for "/data/data/com.termux/files/usr/lib/pulse-12.2/modules/module-roc-sink-input.so"

Thread 1 "pulseaudio" hit Breakpoint 1, 0x0000007fb794cf04 in dlopen () from /system/lib64/libdl.so $55 = 0x7fb6e4b7c0 "/data/data/com.termux/files/usr/lib/pulse-12.2/modules/module-roc-sink-input.so" $56 = 1 E: [pulseaudio] module.c: Failed to open module "module-roc-sink-input". ...


So, it looks pulseaudio tries to load module-roc-sink-input just as any other module. 
But why could dlopen fail on module-roc-sink-input.so?
gavv commented 5 years ago

Tried this - roc module failed to load as before, and module-sles-sink (android's sles sink) also failed to load:

Just to be sure: did you use LD_LIBRARY_PATH="$LD_LIBRARY_PATH:/data/data/com.termux/files/usr/lib" or LD_LIBRARY_PATH="/data/data/com.termux/files/usr/lib"? The later will reset the original LD_LIBRARY_PATH, probably that caused problems with libunwind?

But why could dlopen fail on module-roc-sink-input.so?

I guess other PulseAudio modules don't depend on shared libraries installed in /data/data/com.termux/files/usr/lib/.

gavv commented 5 years ago

You can also try to symlink libroc.so into /data/data/com.termux/files/usr/lib/pulseaudio/ or /data/data/com.termux/files/usr/lib/pulse-12.2/modules/.

If none of these helps, I have no more ideas. Probably the library is not compatible with the daemon because of some compiler options... I'll need to debug it, I think I can do it on next week.

S-trace commented 5 years ago

You can also try to symlink libroc.so into /data/data/com.termux/files/usr/lib/pulseaudio/ or /data/data/com.termux/files/usr/lib/pulse-12.2/modules/.

Tried this, but no luck - still same error.

S-trace commented 5 years ago

Tried to setup roc receiver on another device (Xiaomi MiPad 4/LineageOS 16.0 (Android 9, aarch64)).

Good news - it works right out of the box, like a charm. Bad news - i need it on another device (Sony XZ1 Compact/stock (Android 9, aarch64)). Will try to investigate this issue on the stock Sony firmware.

S-trace commented 5 years ago

Captured logs (3 verbosity levels) from buggy (Sony) and good (Xiaomi) devices. Could You please take a look on it? As far as i see, libroc.so is found, but then rejected by the linker. linker_logs.zip

S-trace commented 5 years ago

Wrote simple dlopen_test.c:

#include <dlfcn.h>
#include <stdio.h>

int main (int argc, char **argv) {
        int success = dlopen(argv[1], RTLD_NOW) != NULL;
        puts(success ? "loaded" : dlerror());
        return success;
}

Built it using gcc -o dlopen_test dlopen_test.c -ldl and then tested all libraries from the output of readelf -a /data/data/com.termux/files/usr/lib/pulse-12.2/modules/module-roc-sink.so|grep NEEDED - all libraries are loadable. Tested module-null-sink.so (passing full path to .so) - loadable. Tried to test /data/data/com.termux/files/usr/lib/module-roc-sink.so and /data/data/com.termux/files/usr/lib/module-roc-sink-input.so and got already known error:

dlopen failed: cannot find "libroc.so" from verneed[1] in DT_NEEDED list for "/data/data/com.termux/files/usr/lib/module-roc-sink.so"

What could be wrong with my phone?

S-trace commented 5 years ago

Figured out the issue. libroc.so was built without SONAME record in Dynamic section. After rebuilding libroc.so with -Wl,-soname,libroc.so in LDFLAGS and module-roc-sink-input.so now works on the Sony phone. Looks like correct SONAME is really necessary on certain Android devices.

gavv commented 5 years ago

Great!

I've added soname support to the develop branch. Could you test that it will now work without setting additional LDFLAGS?

Note: libroc.so now became a symlink, according to the common versioning scheme:

lrwxrwxrwx 1 victor victor   13 Jun 17 15:27 libroc.so -> libroc.so.0.1*
lrwxrwxrwx 1 victor victor   13 Jun 17 15:27 libroc.so.0 -> libroc.so.0.1*
-rwxr-xr-x 1 victor victor 1.9M Jun 17 15:27 libroc.so.0.1*
S-trace commented 5 years ago

Thank you, it works.

gavv commented 5 years ago

Thank you too. Can we close this?

S-trace commented 5 years ago

Yes, we can. But there is one question left: I'm think we can commit updated build scripts to https://github.com/termux/unstable-packages/ but which version should be in the build.sh for roc? 5d00480? Maybe it's a time to publish develop branch as something like 0.1.0_dev0 version?

gavv commented 5 years ago

I think we can tag 0.1.1 and move the rest issues planned to it to 0.1.2. I'll add a minimal travis job for android and prepare a release.

gavv commented 5 years ago

I've pushed the OpenFEC patch (a bit modified) to our fork: https://github.com/roc-project/openfec/commit/b6d32edc75e66d5e036c9346fd0852be09863e8e

gavv commented 5 years ago

Done! https://github.com/roc-project/roc/releases/tag/v0.1.1

gavv commented 5 years ago

I'm think we can commit updated build scripts to https://github.com/termux/unstable-packages/

@S-trace do you have any plans on this?

gavv commented 5 years ago

I'll prepare a PR.

gavv commented 5 years ago

Done :)