noVNC Audio Support in Browsers

[XxAcielxX]

Hello there,

First I'll like to thank you for such an awesome image. I'm doing some test runs using this image (jlesage/baseimage-gui:alpine-3.17-v4) for my container where I would like to play audio from my app (i.e., Audacity).

I'm wondering, what do I need to get the audio to work? I have installed the following packages:

RUN \
    add-pkg audacity ffmpeg font-dejavu \
    dbus-openrc pulseaudio pulseaudio-alsa alsa-plugins-pulse alsa-lib-dev

I see that you have added audify.js, pcm-player.js, script.js for noVNC, but have not been build & installed as ws & audify.

Any help on this will be appreciated.

Regards, XxAcielxX

[jlesage]

Currently, there is no support for audio playing through the web browser. There only way that could work right now is playing audio through the host running the container.

I see that you have added audify.js, pcm-player.js, script.js for noVNC, but have not been build & installed as ws & audify.

Where do you see that ?

[XxAcielxX]

Where do you see that ?

There are inside the container in /opt/noVNC/

I did build & install with:

RUN npm install --prefix /opt/noVNC ws \
    && npm install --prefix /opt/noVNC audify

The run pulseaudio & audify with the help of your startapp.sh:

#!/bin/sh 
exec /usr/bin/pulseaudio &
sleep 1 &
exec /usr/bin/node /opt/noVNC/audify.js &
sleep 1 &
exec /usr/bin/audacity

This is were I'm stuck right now:

[app         ] RtApiAlsa::probeDeviceOpen: unable to synchronize input and output devices.
[app         ] E: [pulseaudio] pstream.c: Falling back to copying full block data over socket
[app         ] E: [pulseaudio] pstream.c: Cannot send block reference with non-registered memfd ID = 3003683793
[app         ] E: [pulseaudio] pstream.c: There's a bug report about this: https://gitlab.freedesktop.org/pulseaudio/pulseaudio/issues/824
[app         ] W: [pulseaudio] main.c: Unable to contact D-Bus: org.freedesktop.DBus.Error.Spawn.ExecFailed: /usr/bin/dbus-launch terminated abnormally without any error message
[app         ] W: [pulseaudio] server-lookup.c: Unable to contact D-Bus: org.freedesktop.DBus.Error.Spawn.ExecFailed: /usr/bin/dbus-launch terminated abnormally without any error message
[app         ] W: [pulseaudio] cli-command.c: stat('/etc/pulse/default.pa.d'): No such file or directory
[app         ] E: [pulseaudio] module.c: Failed to load module "module-console-kit" (argument: ""): initialization failed.
[app         ] E: [pulseaudio] module-console-kit.c: Unable to contact D-Bus system bus: org.freedesktop.DBus.Error.FileNotFound: Failed to connect to socket /var/run/dbus/system_bus_socket: No such file or directory
[app         ] E: [pulseaudio] core-util.c: Failed to connect to system bus: Failed to connect to socket /var/run/dbus/system_bus_socket: No such file or directory
[app         ] W: [pulseaudio] main.c: This program is not intended to be run as root (unless --system is specified).

[jlesage]

Ok so ws and audify is something that you added yourself (it's not part of the official image).

Do you have any pointer/documentation/recipe about what you try to achieve ?

[XxAcielxX]

I took this image https://github.com/novaspirit/Alpine_xfce4_noVNC by Don of Novaspirittech as my reference. If you test run this container, you'll see the the audio works flawlessly for browser(s) vnc.

Though I want to use JWM for the lightweight image for a single app usage.

It is like use pulseaudio to pipe the audio the tcp then pipe it again through the noVNC websocket.

[jlesage]

Ok, this is one of the solutions used to send audio using a side-channel.

Note that I'm working in integrating a such solution into the baseimage.

[XxAcielxX]

That will be great, I'll wait for your update.

Do make it like we could enable audio support when required with an environment. Something like ENABLE_WEB_AUDIO=1, like people who need audio can use it and those who don't, doesn't get the required packages installed.

[kennhardy]

Currently, there is no support for audio playing through the web browser. There only way that could work right now is playing audio through the host running the container.

I see that you have added audify.js, pcm-player.js, script.js for noVNC, but have not been build & installed as ws & audify.

Where do you see that ?

How would I go about for gettings audio from the host machine? Any examples?

[jlesage]

It depends on your application, but adding --device /dev/snd when creating the container might be sufficient.

It's also possible via pulse audio: https://medium.com/@18bhavyasharma/enabling-sound-card-access-in-docker-containers-using-pulseaudio-d52ff1f5eee4

[kennhardy]

Been tinkering with it now. Both PulseAudio and PipeWire seems to be okay options. But seems very compersome to make a unified image that will work on Linux, MacOS and Windows since the prerequirements are too big.

Might look into making web audio work instead. Any pointers to get me started? If i'm able to make it work i'll make a PR.

[jlesage]

Correct, audio on the host is not a solution.

I'm already working on adding support for audio through the web and this is a feature that should be available soon. I just need to figure out why with some browsers the audio lag might increase over time.

[kennhardy]

Do you perhaps have a branch for this release already?

fre 16 feb. 2024 kl. 15:45 skrev Jocelyn Le Sage @.***>:

Correct, audio on the host is not a solution.

I'm already working on adding support for audio through the web and this is a feature that should be available soon. I just need to figure out why with some browsers the audio lag might increase over time.

— Reply to this email directly, view it on GitHub https://github.com/jlesage/docker-baseimage-gui/issues/83#issuecomment-1948516303, or unsubscribe https://github.com/notifications/unsubscribe-auth/ACK6WJZ4QO375MTICIWB36TYT5WJ3AVCNFSM6AAAAAASYERGQGVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTSNBYGUYTMMZQGM . You are receiving this because you commented.Message ID: @.***>

[kennhardy]

NM. Found audio-and-webauth branch. Thank you.

[jlesage]

Audio via web browser is now supported with the latest version of the baseimage.

[kennhardy]

Thank you. Much appreciated!

[XxAcielxX]

Much appreciated, thank you