search

termux / termux-packages

A package build system for Termux.
https://termux.dev
Other
12.98k stars 2.99k forks source link

[TRACKER] Integrating ALSA and Pulseaudio into Termux #821

Closed ackalker closed 7 months ago

ackalker commented 7 years ago

Goals

This issue is for tracking progress on integrating ALSA and Pulseaudio into Termux. Of course, ALSA in Termux will not have direct access to sound hardware on an Android device unless rooted and the vendor uses an ALSA driver in the kernel. The goal is to have a means to communicate with the OpenSL ES subsystem in Android as efficiently as possible. Currently, this is done by streaming a Pulseaudio sink onto the localhost network. In future, it may be possible to write or adapt a Pulseaudio backend driver or an ALSA plugin to talk directly to OpenSL ES, using the so-called "fast mixer", with minimum latency when possible.

Overview of tasks and status

Building and using the required packages

Writing OpenSL ES backend drivers and plugins

Enabling audio related packages to make use of ALSA and/or Pulseaudio

Detailed description

Building and using the required packages

All packages build without problems on my system. They're mostly in a minimal, PoC state, I'm open for discussion on what extra dependencies to include, any tweaks to make.

libpulseaudio is usable with Pulseaudio enabled packages, even without pulseaudio installed:

Known issue: Sound gets muted when device screen turns off or XServer XSDL is not running in the foreground.

pulseaudio is a really bare-bones, minimal affair, no speex (I doubt if it is worth the CPU resources on Android), no ORC (AFAICT it currently only supports an architecture with MMX/SSE instructions anyway). I've also disabled Neon optimization: the build generates assembly which the assembler seems not to be able to handle (it spews errors about unknown registers, q0 among them). I symlinked the shared libraries into $TERMUX_PREFIX/lib to avoid having to write an ugly wrapper. If there are any conflicting filenames, I'd like to hear about them. To get sound output from applications running in Termux, you can adapt and run the following script:

setup-pulseaudio.sh:

#!/data/data/com.termux/files/usr/bin/sh

TERMUX_PREFIX=~/../usr

# Do not shutdown server when idle.
sed -i $TERMUX_PREFIX/etc/pulse/daemon.conf \
    -e '/^; exit-idle-time =/{s/^; //;s/ 20/ -1/}'

# When no sinks are available, Pulseaudio automatically adds a null sink named 'auto_null'.

# Stream the 'auto_null' sink over TCP.
# This can be played by Simple Protocol Player app on Google Play.
sed -i $TERMUX_PREFIX/etc/pulse/default.pa \
    -e '$aload-module module-simple-protocol-tcp source=auto_null.monitor port=12345 record=true'

# In addition, or alternatively,
# add a null sink named 'rtp' and stream it over RTP.
# This can be played by VLC, etc.
# sed -i $TERMUX_PREFIX/etc/pulse/default.pa \
#     -e '/^#load-module module-null-sink sink_name=rtp /s/#//' \
#     -e '/^#load-module module-rtp-send source=rtp.monitor$/s/#//'

To play sound from a TCP stream:

To play sound from an RTP stream:

alsa-lib is pretty complete, I've worked around a problem where the path /etc/asound.conf is hardcoded, which may conflict with system ALSA and isn't useful on non-rooted devices anyway.

alsa-utils: Also pretty complete, not sure if we need all of the utilities. Fun tidbit: alsamixer runs really well in Termux terminal emulator, it's even possible to drag with a finger to change the volume, yay!

alsa-plugins: Only built the Pulseaudio plugin. I'm not sure if we need the samplerate plugin because Pulseaudio and OpenSL ES can both resample audio.

Writing OpenSL ES backend drivers and plugins

pulseaudio is one of the targets for a dedicated OpenSL ES driver, I haven't looked into that yet. There appears to be a (reportedly buggy) backend available, maybe we can adapt that.

alsa-plugins is another target for more direct interaction with OpenSL ES, this time as a (pair of?) plugin(s), a virtual PCM and CTL device.

Enabling audio related packages to make use of ALSA and/or Pulseaudio

cmus runs on Termux. Some of its optional dependencies (support for various file types) still need to be packaged.

Resources

ALSA

Pulseaudio

Android

Third-party libraries

fornwall commented 7 years ago

Nice!

I've also disabled Neon optimization: the build generates assembly which the assembler seems not to be able to handle (it spews errors about unknown registers, q0 among them).

This could be due to clang trying to use its integrated assembler. Does adding -no-integrated-as to CFLAGS fix this and allow neon optimizations?

its-pointless commented 7 years ago

Using gcc fixes that neon issue

ackalker commented 7 years ago

Thanks for your very quick reply! I will look into it.

ackalker commented 7 years ago

Woops, sorry about that! Reopened.

DoktorStein commented 7 years ago

Very interesting the fact that this project is beyond Termux scope. It would be great if eventually it became a official port.

ackalker commented 7 years ago

I hope to push the package build scripts to my fork for testing soon, but I'm hitting a problem with a bogus buffer size when using the ALSA Pulseaudio plugin which causes speaker-test to crash with a Floating point exception. Stay tuned 😄

ackalker commented 7 years ago

@DoktorStein Yes, I agree that it goes beyond Termux as such, but as Termux is such a wonderful project and has a very convenient build system, I think it is only fitting to let Termux be the first to benefit 😉

ackalker commented 7 years ago

Added some resources. slesTestFeedback.cpp in particular looks very interesting, it appears to be some kind of reference test which firmware developers can use to check audio latency on their device before they ship a new ROM. This also explains why it uses some private APIs which are not exposed by the NDK (but can be ported I think).

ackalker commented 7 years ago

I have got cmus running on Termux and playing sound. Still need to package some extra optional dependencies.

pvagner commented 7 years ago

I have just found out a patch adding opensles sink to pulseaudio here: https://github.com/glance-/pulseaudio-android-ndk/blob/master/pulseaudio-patches/0003-WIP-opensl-sink.patch I haven't yet tried to build this I have just looked at it briefly and I have found out it's a different implementation to the one you have mentioned in the first comment here.

pvagner commented 7 years ago

Apparently there is issue with some missing symbols. I'm on LineAgeOS Android 7 and I'm getting it too. I have tried to install glib in case it may help but it appears the missing symbol comes from the android runtime... Here is where I have seen the issue mentioned for the first time... https://github.com/cmus/cmus/issues/688

Grimler91 commented 7 years ago

I'm also experiencing the missing symbol issue on arm android 7.0 with a AOSP-rom.

pvagner commented 7 years ago

hmm. Looking through the list of packages, I assume libandroid-glob is somehow supposed to address this. However I haven't yet understood how to most easily build a package. Originally I was about to build it on the device with termux running however when looking at termux-package repo, I can only see packages and scripts to use in its specific build environment. So I need to learn all this before I can try doing something usefull here. Edit: Linking with libandroid-glob the same way we are linking with liblog should be enough I think. At least this is how it looks like when comparing this with other packages relying on libandroid-glob. I don't have the dev envo configured yet, but I imagine this will be a simple thing to do.

Grimler91 commented 7 years ago

@pvagner I tried building with the option as you suggested and it solved the error! (I'm still not getting audio though so there are more problems. gdb cmus warns about a couple of missing symbols).

Would you like to open a pull request to solve the issue (and add libandroid-glob as a dependency)?

pvagner commented 7 years ago

@Grimler91 have you configured your pulseaudio in termux as proposed by @ackalker via a simple script setup-pulseaudio.sh and are you using either Simple Protocol Player to play back the audio through local network or VLC to play an rtp stream? Apparently no one else besides @ackalker got thus far as we are only discovering all this after a few months. If you have got this far, perhaps you can make the pull request earlier. I am still getting all the stuf setup and I would like to get it building for me before submitting anything. It might take me a few hours or a few days, I am not sure yet how I can manage...

Grimler91 commented 7 years ago

@pvagner Errrm, no I haven't. That could be my problem, I should start reading the instructions before testing stuff.

I tried it again and managed to play local music files using cmus and "simple protocol player" :) Cool that it works!

I'll open a pull request shortly. (And ask if you have any questions about how to setup the build environment)

pvagner commented 7 years ago

@Grimler91 thank you! I've got it building and the sound is playing over here as well.

Grimler91 commented 6 years ago

For anyone using cmus + simpleprotocolplayer: it is possible to start simpleprotolplayer for command line in termux using the intents described in this simpleprotocolplayer issue.

I am now using a small script to allow me to fully control my music playing device when only connected to it over ssh: 

bash-4.4$ more bin/am_protocolplayer.sh
#!/data/data/com.termux/files/usr/bin/bash
if [ $1 == "play" ]; then
    am startservice --user 0 -a com.kaytat.simpleprotocolplayer.action.PLAY -e ip_addr 127.0.0.1 --ei audio_port 12345 --ei sample_rate 44100 --ez stereo true --ei buffer_ms 500 com.kaytat.simpleprotocolplayer/.MusicService
elif [ $1 == "stop" ]; then
    am startservice --user 0 -a com.kaytat.simpleprotocolplayer.action.STOP com.kaytat.simpleprotocolplayer/.MusicService
fi

am_protocolplayer.sh play starts the streaming and am_protocolplayer.sh stop stops the streaming.

twaik commented 6 years ago

Hi! Have working OpenSLES output if pulseaudio writes data to pipe. It can be used instead of simpleprotocolplayer. For now I'm trying to integrate it into module-pipe-sink (using anonymous pipe as buffer). https://gist.github.com/twaik/2c766116950e668989f3497a03b29862 Can anyone help me with that?

twaik commented 6 years ago

Have working OpenSL ES sink. Who I can talk to to test, improve and integrate it to pulseaudio source?

Grimler91 commented 6 years ago

@twaik I assume you want to integrate it into termux's pulseaudio and not the upstream pulseaudio?

Any one can suggest changes to the build scripts. In this case I guess you can suggest a patch to be added to libpulseaudio.

Edit: is it the file you linked to? How should it be integrated into pulseaudio and how can it be tested?

twaik commented 6 years ago

@Grimler91 I have built it on top of that source, but I think it can be integrated into termux-packages repos. I didn't send message to upstream developers because it is not complete yet and because upstream pa have no Android native build support (except this ). It works on my device well but I don't know about running that on other devices. I published the code here. Don't move it to master branch, test first. I need to know restult of a test.

twaik commented 6 years ago

Also these lines need to be added to Makefile.am

...
modlibexec_LTLIBRARIES += \
...
        module-sles-sink.la
...
SYMDEF_FILES = \
...
        module-sles-sink-symdef.h \
...
module_sles_sink_la_SOURCES = modules/sles/module-sles-sink.c
module_sles_sink_la_LDFLAGS = $(MODULE_LDFLAGS) -lOpenSLES
module_sles_sink_la_LIBADD = $(MODULE_LIBADD)
...
twaik commented 6 years ago

Did anyone test that?

its-pointless commented 6 years ago

tried just now won't compile
error: undefined reference to 'initSL' am i using the wrong source as base do i need to use pelya's ? that won't compile with latest ndk ....

twaik commented 6 years ago

@its-pointless sorry, just fixed it

twaik commented 6 years ago

am i using the wrong source as base do i need to use pelya's ?

No, his source have no output to OpenSL, it is a stub one. But my source has.

its-pointless commented 6 years ago

ok its working on the arm device i have i will see if i can get working on aarch64

its-pointless commented 6 years ago

can confirm this works on both arm and aarch64 devices. Nice

twaik commented 6 years ago

@its-pointless Does it work well?

its-pointless commented 6 years ago

In terms of sound im not enough of an audiophile to tell. But i have cmus working. Have to do some scripting so all the sound apps work properly in the same way mpv does. In terms of behaviour i will have to do some more testing later.

its-pointless commented 6 years ago

i will make a pull request with your code added twaik.

twaik commented 6 years ago

@its-pointless OK. Waiting for remarks. Please, tell me if you found something.

twaik commented 6 years ago

About apps using ALSA: try to use this config for alsa

pcm.pulse {
type pulse
}
ctl.pulse {
type pulse
}
pcm.!default {
type pulse
}
ctl.!default {
type pulse
}
its-pointless commented 6 years ago

On my current phone whenever sound is active the "boom sound" becomss activated . This stays on when using pulse audio daemon. Would this have an impact on power consumption? Also alsa compaitibilty is something i should really test for you right?

twaik commented 6 years ago

What is the "boom sound"? I didn't hear about that before. Alsa compatibility is just another function can be used in Termux. Look at the topic name :)

its-pointless commented 6 years ago

its just a mostly software thing from htc. Not significant in of itself but rather indictitive of the software using audio. Its an equalizer that "improves" sound.

twaik commented 6 years ago

I don't know if it can help, but you should try to insert these lines

//change stream type to avoid activating HTC's "boom sound"
SLint32 androidStreamType = SL_ANDROID_STREAM_MEDIA;
//androidStreamType = streamTypeEnum = SL_ANDROID_STREAM_NOTIFICATION;
//androidStreamType = streamTypeEnum = SL_ANDROID_STREAM_RING;
//androidStreamType = streamTypeEnum = SL_ANDROID_STREAM_SYSTEM;
//androidStreamType = streamTypeEnum = SL_ANDROID_STREAM_VOICE;
SLAndroidConfigurationItf androidConfig;
result = (*uriPlayerObject)->GetInterface(s->bqPlayerObject, SL_IID_ANDROIDCONFIGURATION, &androidConfig);checkResult(result);
result = (*androidConfig)->SetConfiguration(androidConfig, SL_ANDROID_KEY_STREAM_TYPE, &androidStreamType, sizeof(SLint32));checkResult(result);

here

twaik commented 6 years ago

I don't know which of SL_ANDROIDSTREAM* activates "boom sound" feature. Need testing.

twaik commented 6 years ago

Maybe "SL_ANDROID_STREAM_SYSTEM" will be the best

its-pointless commented 6 years ago

i doubt it will make much difference. so not really worth getting into it.

its-pointless commented 6 years ago

Iv got it to output from alsa sound out but only aplay and it comes out garbled fartung noise. And it takes a while to get that farting noise ..

twaik commented 6 years ago

For me it works cool. What did you do? Maybe that was an ALSA bug or something... I have installed alsa-utils and libasound2-plugins into chrooted debian environment (Linux Deploy app). I didn't build alsa for android yet.

its-pointless commented 6 years ago

Yes its android alsa...

its-pointless commented 6 years ago

"[ao/alsa] Invalid period size set." When using mpv with alsa audio out

twaik commented 6 years ago

Debian mpv works cool too. I can not reproduce the bug on my device

its-pointless commented 6 years ago

Yeah its likely caused by bionic libc and the restrictions of app,
As in its meant to be our area ... damn it.

twaik commented 6 years ago

I have created repo for my friend. The script inside builds android native Pulseaudio. Can you try to integrate your version of alsa tools and mpv inside it? I can try to fix that

twaik commented 6 years ago

It contains only pulseaudio with its prerequisites build scripts

twaik commented 6 years ago

I have built android native alsa with pulseaudio module included. It works well. I think you were right about the problem in your area.