Add support for the portaudio library

[dubnemo]

I understand you have a Jack wrapper implementation, but native alsa may provide even better performance on the raspberry pi for Jamulus clients leveraging HiFiBerry DAC+ ADC boards.

[npostavs]

I wrote Jamulus' portaudio code so I have an idea about how involved it is ;)

Okay, I actually had forgotten that the non-win32 support was only for the shared lib config, so it was slightly more involved than I realized. But I think the following patch (or git commit here) against the feature branch should work (I tested on my Linux box, though I don't have rPi so I can't guarantee it will work there). Compile with qmake CONFIG+=portaudio and it will use ALSA-only on non-win32 systems.

[PATCH] Allow compiling portaudio with ALSA only

```patch From 7c7ab672a01cae400b689c60201868b8d9ab921b Mon Sep 17 00:00:00 2001 From: Noam Postavsky Date: Sat, 26 Mar 2022 13:15:57 -0400 Subject: [PATCH] Allow compiling portaudio with ALSA only --- Jamulus.pro | 29 ++++++++++++++++++++--------- 1 file changed, 20 insertions(+), 9 deletions(-) diff --git a/Jamulus.pro b/Jamulus.pro index a8fda289..f9b28ab7 100644 --- a/Jamulus.pro +++ b/Jamulus.pro @@ -63,6 +63,8 @@ INCLUDEPATH_OPUS = libs/opus/include \ INCLUDEPATH_PORTAUDIO = libs/portaudio/include libs/portaudio/src/common win32 { INCLUDEPATH_PORTAUDIO += libs/portaudio/src/os/win +} else { + INCLUDEPATH_PORTAUDIO += libs/portaudio/src/os/unix } DEFINES += APP_VERSION=\\\"$$VERSION\\\" \ @@ -763,12 +765,15 @@ win32 { } } else:unix { SOURCES_PORTAUDIO += $$files(libs/portaudio/src/os/unix/*.c) - SOURCES_PORTAUDIO += $$files(libs/portaudio/src/os/hostapi/alsa/*.c) \ - $$files(libs/portaudio/src/os/hostapi/jack/*.c) \ - $$files(libs/portaudio/src/os/hostapi/oss/*.c) + # Does it ever make sense to use OSS? + # $$files(libs/portaudio/src/hostapi/oss/*.c) + SOURCES_PORTAUDIO += $$files(libs/portaudio/src/hostapi/alsa/*.c) + CONFIG(portaudio_jack) { + SOURCES_PORTAUDIO += $$files(libs/portaudio/src/hostapi/jack/*.c) + } } else:macx { SOURCES_PORTAUDIO += $$files(libs/portaudio/src/os/unix/*.c) - SOURCES_PORTAUDIO += $$files(libs/portaudio/src/os/hostapi/coreaudio/*.c) + SOURCES_PORTAUDIO += $$files(libs/portaudio/src/hostapi/coreaudio/*.c) } # I can't figure out how to make the custom compiler stuff work for @@ -1231,8 +1236,12 @@ CONFIG(portaudio) { SOURCES += src/portaudiosound.cpp CONFIG(portaudio_shared_lib) { LIBS += $$libnames(portaudio) - } else:win32 { - DEFINES += PA_USE_ASIO=1 PA_USE_WASAPI=1 PA_USE_WDMKS=1 + } else { + win32 { + DEFINES += PA_USE_ASIO=1 PA_USE_WASAPI=1 PA_USE_WDMKS=1 + } else { + DEFINES += PA_USE_ALSA=1 + } INCLUDEPATH += $$INCLUDEPATH_PORTAUDIO HEADERS += $$HEADERS_PORTAUDIO mingw { @@ -1243,9 +1252,11 @@ CONFIG(portaudio) { SOURCES += $$SOURCES_PORTAUDIO $$SOURCES_CXX_PORTAUDIO } DISTFILES += $$DISTFILES_PORTAUDIO - LIBS += $$libnames(winmm ole32 uuid setupapi) - } else { - error ( "non-shared portaudio only implemented for win32" ) + win32 { + LIBS += $$libnames(winmm ole32 uuid setupapi) + } else { + LIBS += -lasound + } } } -- 2.11.0 ```

[dubnemo]

@npostavs

used qmake CONFIG+=portaudio

it was going so well until this...

g++ -Wl,--as-needed -Wl,-O1 -o Jamulus buffer.o channel.o client.o main.o protocol.o jamcontroller.o server.o serverlist.o serverlogging.o settings.o signalhandler.o socket.o soundbase.o util.o jamrecorder.o creaperproject.o cwavestream.o audiomixerboard.o chatdlg.o clientsettingsdlg.o connectdlg.o clientdlg.o serverdlg.o multicolorled.o levelmeter.o analyzerconsole.o bands.o celt.o celt_decoder.o celt_encoder.o celt_lpc.o cwrs.o entcode.o entdec.o entenc.o kiss_fft.o laplace.o mathops.o mdct.o modes.o pitch.o quant_bands.o rate.o vq.o A2NLSF.o ana_filt_bank_1.o biquad_alt.o bwexpander.o bwexpander_32.o check_control_input.o CNG.o code_signs.o control_audio_bandwidth.o control_codec.o control_SNR.o debug.o decoder_set_fs.o decode_core.o decode_frame.o decode_indices.o decode_parameters.o decode_pitch.o decode_pulses.o dec_API.o encode_indices.o encode_pulses.o enc_API.o apply_sine_window_FLP.o autocorrelation_FLP.o burg_modified_FLP.o bwexpander_FLP.o corrMatrix_FLP.o encode_frame_FLP.o energy_FLP.o find_LPC_FLP.o find_LTP_FLP.o find_pitch_lags_FLP.o find_pred_coefs_FLP.o inner_product_FLP.o k2a_FLP.o LPC_analysis_filter_FLP.o LTP_analysis_filter_FLP.o LTP_scale_ctrl_FLP.o noise_shape_analysis_FLP.o pitch_analysis_core_FLP.o process_gains_FLP.o residual_energy_FLP.o scale_copy_vector_FLP.o scale_vector_FLP.o schur_FLP.o sort_FLP.o warped_autocorrelation_FLP.o wrappers_FLP.o gain_quant.o HP_variable_cutoff.o init_decoder.o init_encoder.o inner_prod_aligned.o interpolate.o lin2log.o log2lin.o LPC_analysis_filter.o LPC_fit.o LPC_inv_pred_gain.o LP_variable_cutoff.o NLSF2A.o NLSF_decode.o NLSF_del_dec_quant.o NLSF_encode.o NLSF_stabilize.o NLSF_unpack.o NLSF_VQ.o NLSF_VQ_weights_laroia.o NSQ.o NSQ_del_dec.o pitch_est_tables.o PLC.o process_NLSFs.o quant_LTP_gains.o resampler.o resampler_down2.o resampler_down2_3.o resampler_private_AR2.o resampler_private_down_FIR.o resampler_private_IIR_FIR.o resampler_private_up2_HQ.o resampler_rom.o shell_coder.o sigm_Q15.o sort.o stereo_decode_pred.o stereo_encode_pred.o stereo_find_predictor.o stereo_LR_to_MS.o stereo_MS_to_LR.o stereo_quant_pred.o sum_sqr_shift.o tables_gain.o tables_LTP.o tables_NLSF_CB_NB_MB.o tables_NLSF_CB_WB.o tables_other.o tables_pitch_lag.o tables_pulses_per_block.o table_LSF_cos.o VAD.o VQ_WMat_EC.o analysis.o mlp.o mlp_data.o opus.o opus_decoder.o opus_encoder.o repacketizer.o portaudiosound.o qrc_resources.o moc_channel.o moc_client.o moc_protocol.o moc_jamcontroller.o moc_server.o moc_serverlist.o moc_settings.o moc_socket.o moc_soundbase.o moc_testbench.o moc_util.o moc_jamrecorder.o moc_creaperproject.o moc_signalhandler.o moc_audiomixerboard.o moc_chatdlg.o moc_clientsettingsdlg.o moc_connectdlg.o moc_clientdlg.o moc_serverdlg.o moc_levelmeter.o moc_analyzerconsole.o moc_multicolorled.o   -lasound -lpthread /usr/lib/aarch64-linux-gnu/libQt5Widgets.so /usr/lib/aarch64-linux-gnu/libQt5Gui.so /usr/lib/aarch64-linux-gnu/libQt5Network.so /usr/lib/aarch64-linux-gnu/libQt5Xml.so /usr/lib/aarch64-linux-gnu/libQt5Concurrent.so /usr/lib/aarch64-linux-gnu/libQt5Core.so -lGL   
/usr/bin/ld: portaudiosound.o: in function `CSound::GetNumInputChannels() [clone .localalias]':
portaudiosound.cpp:(.text+0x28): undefined reference to `Pa_GetDeviceInfo'
/usr/bin/ld: portaudiosound.o: in function `CSound::GetNumOutputChannels() [clone .localalias]':
portaudiosound.cpp:(.text+0x58): undefined reference to `Pa_GetDeviceInfo'
/usr/bin/ld: portaudiosound.o: in function `CSound::UnloadCurrentDriver()':
portaudiosound.cpp:(.text+0x88): undefined reference to `Pa_CloseStream'
/usr/bin/ld: portaudiosound.cpp:(.text+0xa0): undefined reference to `Pa_Terminate'
/usr/bin/ld: portaudiosound.o: in function `CSound::Start()':
portaudiosound.cpp:(.text+0x214): undefined reference to `Pa_StartStream'
/usr/bin/ld: portaudiosound.o: in function `CSound::Stop()':
portaudiosound.cpp:(.text+0x244): undefined reference to `Pa_StopStream'
/usr/bin/ld: portaudiosound.o: in function `CSound::SetLeftInputChannel(int)':
portaudiosound.cpp:(.text+0x2b4): undefined reference to `Pa_GetDeviceInfo'
/usr/bin/ld: portaudiosound.o: in function `CSound::SetRightInputChannel(int)':
portaudiosound.cpp:(.text+0x324): undefined reference to `Pa_GetDeviceInfo'
/usr/bin/ld: portaudiosound.o: in function `CSound::SetLeftOutputChannel(int)':
portaudiosound.cpp:(.text+0x394): undefined reference to `Pa_GetDeviceInfo'
/usr/bin/ld: portaudiosound.o: in function `CSound::SetRightOutputChannel(int)':
portaudiosound.cpp:(.text+0x404): undefined reference to `Pa_GetDeviceInfo'
/usr/bin/ld: portaudiosound.o: in function `CSound::InitPa()':
portaudiosound.cpp:(.text+0x584): undefined reference to `Pa_Initialize'
/usr/bin/ld: portaudiosound.cpp:(.text+0x594): undefined reference to `Pa_GetHostApiCount'
/usr/bin/ld: portaudiosound.cpp:(.text+0x5cc): undefined reference to `Pa_GetHostApiInfo'
/usr/bin/ld: portaudiosound.cpp:(.text+0x650): undefined reference to `Pa_GetErrorText'
/usr/bin/ld: portaudiosound.o: in function `CSound::ReinitializeDriver(int, int)':
portaudiosound.cpp:(.text+0x95c): undefined reference to `Pa_GetDeviceInfo'
/usr/bin/ld: portaudiosound.cpp:(.text+0x968): undefined reference to `Pa_GetDeviceInfo'
/usr/bin/ld: portaudiosound.cpp:(.text+0x978): undefined reference to `Pa_CloseStream'
/usr/bin/ld: portaudiosound.cpp:(.text+0xa2c): undefined reference to `Pa_OpenStream'
/usr/bin/ld: portaudiosound.cpp:(.text+0xa60): undefined reference to `Pa_GetErrorText'
/usr/bin/ld: portaudiosound.cpp:(.text+0xaa8): undefined reference to `Pa_GetLastHostErrorInfo'
/usr/bin/ld: portaudiosound.o: in function `CSound::~CSound()':
portaudiosound.cpp:(.text+0xcc4): undefined reference to `Pa_Terminate'
/usr/bin/ld: portaudiosound.o: in function `CSound::GetPaApiNames()':
portaudiosound.cpp:(.text+0xd98): undefined reference to `Pa_Initialize'
/usr/bin/ld: portaudiosound.cpp:(.text+0xda4): undefined reference to `Pa_GetHostApiCount'
/usr/bin/ld: portaudiosound.cpp:(.text+0xdd4): undefined reference to `Pa_GetHostApiInfo'
/usr/bin/ld: portaudiosound.cpp:(.text+0xe48): undefined reference to `Pa_Terminate'
/usr/bin/ld: portaudiosound.cpp:(.text+0xe74): undefined reference to `Pa_Terminate'
/usr/bin/ld: portaudiosound.cpp:(.text+0xe9c): undefined reference to `Pa_Terminate'
/usr/bin/ld: portaudiosound.o: in function `CSound::Init(int)':
portaudiosound.cpp:(.text+0xef8): undefined reference to `Pa_GetHostApiInfo'
/usr/bin/ld: portaudiosound.o: in function `CSound::CSound(void (*)(CVector<short>&, void*), void*, QString const&, bool, QString const&, QString const&)':
portaudiosound.cpp:(.text+0x10a8): undefined reference to `Pa_HostApiDeviceIndexToDeviceIndex'
/usr/bin/ld: portaudiosound.cpp:(.text+0x10b0): undefined reference to `Pa_GetDeviceInfo'
/usr/bin/ld: portaudiosound.cpp:(.text+0x10ec): undefined reference to `Pa_HostApiDeviceIndexToDeviceIndex'
/usr/bin/ld: portaudiosound.cpp:(.text+0x10f4): undefined reference to `Pa_GetDeviceInfo'
collect2: error: ld returned 1 exit status
make: *** [Makefile:1109: Jamulus] Error 1

[dubnemo]

trying qmake 'CONFIG+=portaudio portaudio_shared_lib'

[npostavs]

it was going so well until this...

Hmm, looks like it's failing to link portaudio sources, I don't understand why.

[dubnemo]

This did not return any errors BUT on RPi4 Jamulus was not added to the menu.

qmake 'CONFIG+=portaudio portaudio_shared_lib' 
make

I am trying again..

Compiling.md definitely requires this for the RPi4: sudo apt install portaudio19-dev

Once PortAudio 19.8 is available, I will try to build a slim MVP package.

[dubnemo]

I suspect this is due to the PortAudio 19.7 package I installed:

[npostavs]

I suspect this is due to the PortAudio 19.7 package I installed:

Oh, I forgot to mention you have to run with Jamulus --api ALSA.

[dubnemo]

I will have time to fully test this Wednesday evening Central (us).

After editing jamulus.desktop, I have now have the Jamulus client running and connected to my local server in my studio basement. The previous performance with the Hi-Fi Berry ADC-DAC was ~80ms total roundtrip, and this is about 1/2.

I am not sure about the Pulse device, but again, I will test this out.

pi@raspberrypi:/usr/local/share/applications $ sudo nano jamulus.desktop
pi@raspberrypi:/usr/local/share/applications $ cat jamulus.desktop
[Desktop Entry]
Name=Jamulus
Comment=Jam Session
Comment[fr]=Séance de bœuf
Comment[sv]=Musikaliska jamsessioner över Internet
GenericName=Internet Jam Session Software
GenericName[fr]=Logiciel de séance de bœuf sur Internet
GenericName[es]=Software para Jam Sessions por Internet
GenericName[pt]=Software para Jam Sessions pela Internet
GenericName[nl]=Software voor jamsessies over internet
GenericName[sk]=Softvér na džemovanie cez internet
GenericName[sv]=Mjukvara för Jam Sessioner över Internet
Exec=Jamulus --api ALSA
Icon=jamulus
Terminal=false
Type=Application
Categories=AudioVideo;Audio;Mixer;Qt;
Keywords=jam;live;online;music;conference;

When I select the Hi-Fi Berry from the Device drop-down directly, it is throwing an exception. But that may be a configuration issue.

[jujudusud]

Nice ! @npostavs: do you need X86_64 Linux and windows 10 64 bits testers ?

I want to test Windows WASAPI to use it instead of asio4all.

P.S.: No need to use OSS server in 2022 from my point of view.

[dubnemo]

@npostavs: I woke up in the middle of the night thinking the Hi-Fi Berry device should work, as there isn't much to configure. I only edited the /boot/config.txt

# use the headphone output jack on the RPi4
dtparam=audio=on
# enable DAC+ ADC Pro
dtoverlay=hifiberry-dacplusadcpro

#Disable Wifi; use eth0 for Jamulus only
dtoverlay=disable-wifi 
# disable bluetooth
dtoverlay=disable-bt

I have not created ~/.asoundrc (my user only) or /etc/asound.conf (all users) files.

There may have an interface mismatch with the number of channels(?).

[npostavs]

I want to test Windows WASAPI to use it instead of asio4all.

You can give it a shot, although I found the results disappointing on my hardware (run Jamulus with --api WASAPI to activate). Note that you can also try KoordASIO instead of asio4all; it uses WASAPI via PortAudio too (and likewise doesn't seem to work well with my hardware).

There may have an interface mismatch with the number of channels(?).

Hmm, I'm not sure what's triggerring that error.

[dubnemo]

@npostavs I thought you'd like to know that I am getting the same error with the channel interface.

Just to be clear the configuration

pi@raspberrypi:~ $ cat /proc/asound/cards
 0 [Headphones     ]: bcm2835_headpho - bcm2835 Headphones
                      bcm2835 Headphones
 1 [vc4hdmi0       ]: vc4-hdmi - vc4-hdmi-0
                      vc4-hdmi-0
 2 [vc4hdmi1       ]: vc4-hdmi - vc4-hdmi-1
                      vc4-hdmi-1
 3 [sndrpihifiberry]: HifiberryDacpAd - snd_rpi_hifiberry_dacplusadcpro
                      snd_rpi_hifiberry_dacplusadcpro

pi@raspberrypi:~ $ cat /proc/asound/devices
  0: [ 0]   : control
 16: [ 0- 0]: digital audio playback
 32: [ 1]   : control
 33:        : timer
 48: [ 1- 0]: digital audio playback
 64: [ 2]   : control
 80: [ 2- 0]: digital audio playback
 96: [ 3]   : control
112: [ 3- 0]: digital audio playback
120: [ 3- 0]: digital audio capture

Works in Audacity:

But I get the error when selecting the device in the Jamulus client:

Really unrelated, but I am unclear why PulseAudio daemon is referenced, even when I remove it:

pi@raspberrypi:~ $ sudo apt-get purge pulseaudio
Reading package lists... Done
Building dependency tree... Done
Reading state information... Done
The following packages were automatically installed and are no longer required:
  libpulsedsp pulseaudio-utils rtkit
Use 'sudo apt autoremove' to remove them.
The following packages will be REMOVED:
  lxplug-volumepulse* pulseaudio* pulseaudio-module-bluetooth*
0 upgraded, 0 newly installed, 3 to remove and 48 not upgraded.
After this operation, 6,919 kB disk space will be freed.
Do you want to continue? [Y/n] Y
(Reading database ... 95430 files and directories currently installed.)
Removing lxplug-volumepulse (0.12) ...
Removing pulseaudio-module-bluetooth (14.2-2+rpi1) ...
Removing pulseaudio (14.2-2+rpi1) ...
Processing triggers for man-db (2.9.4-2) ...
(Reading database ... 95179 files and directories currently installed.)
Purging configuration files for pulseaudio (14.2-2+rpi1) ...
Processing triggers for dbus (1.12.20-2) ...

oddly it still shows up when I start the Jamulus client:

pi@raspberrypi:~ $ ps -ef | grep pulse
pi           653     632  1 22:35 ?        00:00:27 /usr/bin/pulseaudio --daemonize=no --log-target=journal
pi          3136    1613  0 23:09 pts/0    00:00:00 grep --color=auto pulse

And the PulseAudio sound server interface seems to be working (I will check my server recording). If this daemon can be bypassed and go direct via PortAudio like Audacity, I suspect the total delay can be further reduced.

[dubnemo]

PortAudio seems to use PulseAudio sound server. I went into Audacity again, then killed pid 632, and the RPi4 screen went blank and restarted with the login screen.

[elliotclee]

Pulseaudio is one of the daemons that get started with the login session, so it's not surprising that the session ended when you killed it.

Now that you've removed the pulseaudio package and killed the process, can you login again and get Jamulus to run?

Elliot

---

From: Scott T Nieman @.> Sent: Thursday, March 31, 2022 9:02:15 AM To: jamulussoftware/jamulus @.> Cc: elliotclee @.>; Mention @.> Subject: Re: [jamulussoftware/jamulus] Add support for the portaudio library (#116)

PortAudio seems to use PulseAudio sound server. I went into Audacity again, then killed pid 632, and the RPi4 screen went blank and restarted with the login screen.

— Reply to this email directly, view it on GitHubhttps://github.com/jamulussoftware/jamulus/issues/116#issuecomment-1084548761, or unsubscribehttps://github.com/notifications/unsubscribe-auth/AJNWQFHJN5WKIWVVGONZTYLVCWO5PANCNFSM4ML7LMQA. You are receiving this because you were mentioned.Message ID: @.***>

[dubnemo]

@elliotclee yes, you are correct. It is not spawned by PortAudio. Jamulus client saw PulseAudio and that was able to communicate to the HiFi Berry card in my earlier screenshots, hence why it was able to start. Now after restart of the RPi4, I now see this on startup of Jamulus client.

@npostavs do you think there a PortAudio callback that is not set properly? Audacity still works after uninstall of PulseAudio too. Is there a way to turn on fine logging in the client via a cli parameter?

[npostavs]

I haven't added a logging option; I'm not really what there is to log.

You could try fiddling with the setting of paInputParams.suggestedLatency in src/portaudiosound.cpp, I did find that I got some confusing errors on Windows when it was set to the wrong value (and it's still unclear to me how to know the right value).

Might also be worth investigating whether any the functions in http://files.portaudio.com/docs/v19-doxydocs/pa__linux__alsa_8h.html helps. At least PaAlsa_EnableRealtimeScheduling seems relevant.

[dubnemo]

@npostavs
Curious what is in devInfo for the Hi Fi Berry, as it has a 35mm stereo input and two RCA outputs available out of the box, but it appears there are balanced inputs and outputs available on the board with soldering. From above:

112: [ 3- 0]: digital audio playback
120: [ 3- 0]: digital audio capture

One of the above errors showed 'invalid number of channels'.

[dubnemo]

It seems more on the device initialization, versus streams.

http://files.portaudio.com/docs/v19-doxydocs/api_overview.html Although each Device conceptually belongs to a specific Host API, most PortAudio functions and data structures refer to Devices using a global, Host API-independent index of type PaDeviceIndex an integer of that ranges between zero and Pa_GetDeviceCount() - 1. The reasons for this are partly historical but it also makes it easy for applications to ignore the Host API abstraction and just work with Devices and Streams.

If you want to enumerate Devices belonging to a particular Host API you can count between 0 and PaHostApiInfo::deviceCount - 1. You can convert this Host API-specific index value to a global PaDeviceIndex value by calling Pa_HostApiDeviceIndexToDeviceIndex().

===== deviceCount -1 may be relevant.

Do you really want to iterate on the api count or the device count? for ( PaHostApiIndex i = 0; i < apiCount; i++ ); ... selectedApiIndex = apiIndex;

selectedApiIndex is passed as an argument to Pa_HostApiDeviceIndexToDeviceIndex

PaDeviceIndex outDevIndex = Pa_HostApiDeviceIndexToDeviceIndex ( selectedApiIndex, j );