State of the library in comparison to OpenAL Soft?

[raysan5]

Hi @mackron, congrats for such an amazing lib, I've been looking for a library like this for long time now.

I'm thinking about replacing current raylib audio module backend (OpenAL Soft) by mini_al and I would like to know how mini_al compares to OpenAL Soft in features... Actually, I'm mostly just using audio playback for complete loaded sounds and music streaming.

My final goal would be to remove any external library dependency for raylib, I mean, include with raylib basecode all required dependencies.

[mackron]

Thanks!

The first thing to keep in mind is that mini_al is not a 3D audio API, so if that's something you need then it won't be suitable (or you'll need to do your own mixing). Besides that, there's still a bit of work to do before it's on par with OpenAL-Soft:

• I still need to get a CoreAudio back end done for macOS and iOS. I don't own a Mac, though, and they're kind of expensive where I live so I don't know when I'll get around to that
  • There's an OpenAL back end in place, but it's never been tested on Mac
• PulseAudio and JACK back ends are not currently supported, but they're in the pipeline
• mini_al has a feature where it does automatic format conversion between the format requested by the application and the format supported by the internal back end device. This conversion pipeline needs a bit of work in terms of both optimizations and features:
  • The sample rate conversion algorithm is just a dumb, low quality linear algorithm
  • Channel conversion (stereo to 5.1, for example) is very basic at the moment - it simply drops excess channels or inserts silence into extra channels. I want to add support for something a bit smarter where it properly blends/averages channels instead.

I took a quick look at your code. You'd need to do a bit of work to implement your own mixer. mini_al does not do any sound/source/buffer management like OpenAL, so all of the logic for mixing the audio data of currently playing sounds and music will need to be done manually. Is volume and pitch the only effects you apply to sounds and music? From what I can tell, it looks like you've actually got fairly simple audio requirements so your own mixer wouldn't be too difficult. If you like, I can probably get something quick and dirty up and running for you as a proof of concept? If I can get that working, and then have somebody test/fix the OpenAL back end on Mac, it may actually work...

I also noticed you're using my dr_flac library. Nice :) 👍

[raysan5]

Hi @mackron! Thank you very much for the quick answer! :D

I still need to get a CoreAudio back end done for macOS and iOS...

That's not a big issue, I can keep my OpenAL Soft backend on those platforms... I neither have a Mac to test it, all raylib testing on macOS has been done by users.

PulseAudio and JACK back ends are not currently supported, but they're in the pipeline

No problem.

mini_al has a feature where it does automatic format conversion...

That's great, actually, I also tried to implement some functions to format wave data... but very bad quality...

You'd need to do a bit of work to implement your own mixer. mini_al does not do any sound/source/buffer management like OpenAL, so all of the logic for mixing the audio data of currently playing sounds and music will need to be done manually.

Yeah, that's what I saw... probably that's the difficult part... dealing with sounds/source/buffers was quite easy...

Is volume and pitch the only effects you apply to sounds and music?

Yes, actually I exposed that for raylib users because it came with OpenAL Soft...

If you like, I can probably get something quick and dirty up and running for you as a proof of concept?

Wow! That would be great! :D

I also noticed you're using my dr_flac library.

Yeah, it's great! Been following your libs for some time now.

raylib is a simple and easy-to-use library primary intended for education, prototyping and small games so, no need for advanced audio features, there are bigger libraries out there intended for bigger projects. :)

[mackron]

I've done some work on this and it seems to work fine so far. You can follow my progress here: https://github.com/mackron/raylib/tree/dr/mini_al (the dr/mini_al branch).

I've got Sound, AudioStream and Music working, but I've not tested it thoroughly. It's very quick and dirty and probably has a few bugs, but it shows that mini_al can work. I can polish it up a little bit if you decide mini_al integration might be worth pursuing.

A few things:

• I haven't got the pitch effect working yet, mainly because I just don't know how to do it off the top of my head...
• The output volume seems to be slightly louder with mini_al... Not fully sure why, but suspecting it's got something to do with different algorithms being used between mini_al and OpenAL for format conversion.
• The implementation of mini_al needs to be put into a separate translation unit to audio.c because it causes conflicts with Win32 APIs like CloseWindow().
• I've left the OpenAL implementation in place. Toggle it with the USE_MINI_AL define at the top of audio.c.
• mini_al does mixing asynchronously on a separate thread so you'll need to keep thread-safety in mind if you use mini_al. I've got some basic stuff in there that should work, but it's hardly been tested. Just keep that in mind...
  • The synchronization technique I use makes it not "real-time", but I don't think that matters for this project (and I'd be surprised if OpenAL-Soft is real-time anyway, but I could be wrong).
• From what I can tell it looks like mini_al uses less memory.
• I've not tested this, but I think mini_al will be much more CPU efficient than OpenAL-Soft because it's software mixing is so much simpler.
• With a relatively small amount of work you should pretty easily be able to support any format/channels/rate combinations for both playback output and wav/flac/ogg files. Anything that's supported by mini_al should work fine.

I also have some suggestions for improving your API design in a few places, but I think I'll raise a separate issue about that one...

Have fun playing around with this. Let me know if you have any issues!

[raysan5]

Hi @mackron! Amazing work! Thank you very much for your time and your effort on this feature!

About your comments:

I can polish it up a little bit if you decide mini_al integration might be worth pursuing.

Undoubtly it worths it! Replacing such a big dependency as is OpenAL Soft by a more lightweight, portable and faster alternative is a goal!

I haven't got the pitch effect working yet, mainly because I just don't know how to do it off the top of my head...

No worries, it's not crucial... I think it's not that easy, it involves some Fourier Transformation... Here I found a good reference and some code.

The output volume seems to be slightly louder with mini_al... Not fully sure why, but suspecting it's got something to do with different algorithms being used between mini_al and OpenAL for format conversion.

No big issue, it can be adjusted to a lower base value.

The implementation of mini_al needs to be put into a separate translation unit to audio.c because it causes conflicts with Win32 APIs like CloseWindow().

Yeah, I know... the infamous windows.h, I've dealt with that in the past just defining the myself the required parts of windows.h but it seems in this case it requires lots of typedefs... well, no problem.

I've left the OpenAL implementation in place. Toggle it with the USE_MINI_AL define at the top of audio.c.

That's great, allowing the user to choose between both as a transition step is the best idea!

mini_al does mixing asynchronously on a separate thread so you'll need to keep thread-safety in mind if you use mini_al. I've got some basic stuff in there that should work, but it's hardly been tested. Just keep that in mind...

Ok.

The synchronization technique I use makes it not "real-time", but I don't think that matters for this project (and I'd be surprised if OpenAL-Soft is real-time anyway, but I could be wrong).

Ok.

From what I can tell it looks like mini_al uses less memory.

That's great!

I've not tested this, but I think mini_al will be much more CPU efficient than OpenAL-Soft because it's software mixing is so much simpler.

That's really great! :D

With a relatively small amount of work you should pretty easily be able to support any format/channels/rate combinations for both playback output and wav/flac/ogg files. Anything that's supported by mini_al should work fine.

Amazing!

I also have some suggestions for improving your API design in a few places, but I think I'll raise a separate issue about that one...

Don't hessitate to open an issue or just send the improvements with a pull request.

Have fun playing around with this. Let me know if you have any issues!

I tried compiling it with MinGW32 (GCC 5.3.0) and I got some errors:

In file included from external/mini_al.c:4:0:
external/mini_al.h:2720:3: error: conflicting types for 'WAVEFORMATEXTENSIBLE'
 } WAVEFORMATEXTENSIBLE;
   ^
In file included from external/mini_al.h:2683:0,
                 from external/mini_al.c:4:
c:\raylib\mingw\include\mmreg.h:48:3: note: previous declaration of 'WAVEFORMATEXTENSIBLE' was here
 } WAVEFORMATEXTENSIBLE,*PWAVEFORMATEXTENSIBLE;
   ^
In file included from external/mini_al.c:4:0:
external/mini_al.h:2826:25: fatal error: audioclient.h: No such file or directory

You can send me a pull request to my develop branch if you want, I will need some time to review all the changes and understand how everything works but after a quick look it seems quite clear and comprehensive.

[mackron]

Thanks for that MinGW compiler error - I'll get that fixed up. I'll do a bit of polish and more testing before I doing a pull request. Not sure when that'll be, but I won't forget :)

The complexity with pitch shifting is keeping the original sound the same length. I can easily change the pitch by simply running every frame through a sample rate converter, but that lengthens or shortens the sound, depending on the direction of the shift. I'll take a look at those references you provided.

My original intent for the USE_MINI_AL macro was mainly just for testing and safety in case mini_al doesn't work out too well. In the future when the mini_al version matures I would just remove the OpenAL version entirely just to keep it simpler.

[raysan5]

I'll do a bit of polish and more testing before I doing a pull request. Not sure when that'll be, but I won't forget :)

Great! No hurries, take your time! :)

[mackron]

I've updated the dr/mini_al branch with a fix for the first compiler error, but regarding this error:

external/mini_al.h:2826:25: fatal error: audioclient.h: No such file or directory

That's one of the headers for the WASAPI backend. You need to update your Win32 API. Look at the "w32api" download on this page to get you started. In the short term, add "#define MAL_NO_WASAPI" before the implementation of mini_al (in mini_al.c). Let me know how that goes!

As a side note, I'm going to start using the "__has_include" statement with GCC (and any other compiler that supports it) to cleanly exclude backends when the compiler does not have the necessary development packages installed.

[raysan5]

Hi @mackron, thanks for the quick solution!

I'm updating my MinGW headers and libs, it seems some packages were missing.

[mackron]

I just pushed some changes to branch dr/mini_al which adds support for pitch shifting. The mini_al backend should now be at feature parity. Were you ever able to get it working on your end? Pull request coming soon.

[raysan5]

Ok, managed to compile it and it works flawlessly. 😄

Just tried with mingw32 updated to latest available version (GCC 6.3.0); neither MinGW32 or TDM-GCC-32 toolchains include the required libraries for WASAPI and DIRECT SOUND backends, so, I had to comment those lines to force WinMM backend; also tried to define it in audio.c:

#define MAL_NO_WASAPI
#define MAL_NO_DSOUND
#include "external/mini_al.h"

but it didn't work, MAL_WIN32 forces support on mini_al.h, it seems some check is not working...

When compiling mini_al.c module with GCC 6.3.0 and -Wall got some warnings, nothing significant but just listing them here:

gcc -O2 -c external/mini_al.c -Wall -I.
Process started >>>
In file included from external/mini_al.c:4:0:
external/mini_al.h: In function 'mal_device_init':
external/mini_al.h:8738:10: warning: cast from pointer to integer of different size [-Wpointer-to-int-cast]
     if (((mal_uint64)pDevice % sizeof(pDevice)) != 0) {
          ^
external/mini_al.h: In function 'mal_src_read_frames_ex':
external/mini_al.h:9357:23: warning: variable 'algorithm' set but not used [-Wunused-but-set-variable]
     mal_src_algorithm algorithm = pSRC->config.algorithm;
                       ^~~~~~~~~
external/mini_al.h: In function 'mal_pcm_f32_to_u8':
external/mini_al.h:10459:9: warning: dereferencing type-punned pointer will break strict-aliasing rules [-Wstrict-aliasing]
         s = ((*((int*)&x)) & 0x80000000) >> 31;
         ^
external/mini_al.h: In function 'mal_pcm_f32_to_s16':
external/mini_al.h:10475:9: warning: dereferencing type-punned pointer will break strict-aliasing rules [-Wstrict-aliasing]
         s = ((*((int*)&x)) & 0x80000000) >> 31;
         ^
external/mini_al.h: In function 'mal_pcm_f32_to_s24':
external/mini_al.h:10490:9: warning: dereferencing type-punned pointer will break strict-aliasing rules [-Wstrict-aliasing]
         s = ((*((int*)&x)) & 0x80000000) >> 31;
         ^
external/mini_al.h: In function 'mal_pcm_f32_to_s32':
external/mini_al.h:10505:9: warning: dereferencing type-punned pointer will break strict-aliasing rules [-Wstrict-aliasing]
         s = ((*((int*)&x)) & 0x80000000) >> 31;
         ^
In file included from external/mini_al.c:4:0:
At top level:
external/mini_al.h:2538:13: warning: 'MAL_GUID_KSDATAFORMAT_SUBTYPE_IEEE_FLOAT' defined but not used [-Wunused-variable]
 static GUID MAL_GUID_KSDATAFORMAT_SUBTYPE_IEEE_FLOAT = {0x00000003, 0x0000, 0x0010, {0x80, 0x00, 0x00, 0xaa, 0x00, 0x38, 0x9b, 0x71}};
             ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
external/mini_al.h:2537:13: warning: 'MAL_GUID_KSDATAFORMAT_SUBTYPE_PCM' defined but not used [-Wunused-variable]
 static GUID MAL_GUID_KSDATAFORMAT_SUBTYPE_PCM        = {0x00000001, 0x0000, 0x0010, {0x80, 0x00, 0x00, 0xaa, 0x00, 0x38, 0x9b, 0x71}};
             ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
<<< Process finished. (Exit code 0)

Again, thank you very much for all the hard work! It's trully amazing!

[mackron]

I'll get those issues sorted out. Thanks.

Strange that they don't come with dsound.h and audioclient.h... I've never known a distribution of MinGW to not include headers for DirectSound...

[mackron]

I'm just looking at these compiler issues you're having, and indeed it looks like the old 32-bit versions of MinGW do not include dsound.h nor audioclient.h (why?!). The 64-bit version of TDM-GCC definitely includes them, though, because that's what I use for GCC on Windows. Also, you shouldn't comment out those MAL_SUPPORTS_DSOUND and MAL_SUPPORTS_WASAPI lines.

I also just wanted to clarify something. When you say #define-ing MAL_NO_DSOUND and MAL_NO_WASAPI didn't work - what exactly didn't work? What errors were you getting? I just tried on that same version of MinGW you're building against and it works fine (except for those other warnings you mentioned).

[raysan5]

Sorry, my fault, double checked it... I was defining on audio.c:

#define MAL_NO_WASAPI
#define MAL_NO_DSOUND
#include "external/mini_al.h"

but not on mini_al.c module compilation. You're right, it works fine.

Yeah, I also checked MinGW-w64 and it comes with all required libraries and headers, actually, the different MinGW32 versions also include the libdsound.a library but not the header... weird.

Probably I should use MinGW-w64 32bit version of the package, more updated and complete than the old MinGW32 project...

[r-lyeh-archived]

I can confirm that years ago, my mingw setup did not include any DirectX header/library at all. It changed at some point later, for good.

[mackron]

I've updated the dr/mini_al branch with fixes for those warnings.

I can confirm that years ago, my mingw setup did not include any DirectX header/library at all.

I had been using old MinGW (not MinGW-w64) for ages and never realized it didn't have dsound.h!

[raysan5]

I've been trying raylib with mini_al in multiple platforms and those are the results:

PLATFORM_DESKTOP (Windows 10 64bit)

Everything works great.

PLATFORM_RPI (Raspbian Stretch)

Compilation worked ok, just required libsound2-dev ALSA backend and program linkage with ld, as specified in instructions. Compilation output:

gcc -c external/mini_al.c -O1 -Wall -std=c99 -D_DEFAULT_SOURCE -fgnu89-inline -Wno-missing-braces -I. -Iexternal -Iexternal/glfw/include -I/opt/vc/include -I/opt/vc/include/interface/vmcs_host/linux -I/opt/vc/include/interface/vcos/pthreads -I/opt/vc/include/interface/vcos/pthreads -DPLATFORM_RPI
In file included from external/mini_al.c:6:0:
external/mini_al.h: In function ‘mal_device_init’:
external/mini_al.h:8738:10: warning: cast from pointer to integer of different size [-Wpointer-to-int-cast]
     if (((mal_uint64)pDevice % sizeof(pDevice)) != 0) {
          ^
external/mini_al.h: In function ‘mal_src_read_frames_ex’:
external/mini_al.h:9357:23: warning: variable ‘algorithm’ set but not used [-Wunused-but-set-variable]
     mal_src_algorithm algorithm = pSRC->config.algorithm;

Unfortunately, when running audio examples I got a Segmentation fault, audio_sound_loading is the only one that goes beyond the first frame and crashes on PlaySound().

PLATFORM_ANDROID (Android NDK API 16)

Everything compiles fine, APK is generated fine but when running no audio is played. No crash or weird behaviour, just no audio. Compilation output:

C:/android_toolchain_arm_api16/bin/arm-linux-androideabi-gcc -c external/mini_al.c -O1 -Wall -std=c99 -D_DEFAULT_SOURCE -fgnu89-inline -Wno-missing-braces -march=armv7-a -mfloat-abi=softfp -mfpu=vfpv3-d16 -ffunction-sections -funwind-tables -fstack-protector-strong -fPIC -Wa,--noexecstack -Wformat -Werror=format-security -no-canonical-prefixes -DANDROID -DPLATFORM_ANDROID -D__ANDROID_API__=16 -I. -Iexternal -Iexternal/glfw/include -IC:/android_toolchain_arm_api16/sysroot/usr/include -Iexternal/android/native_app_glue -DPLATFORM_ANDROID 
In file included from external/mini_al.c:6:0:
external/mini_al.h: In function 'mal_device_init__opensl':
external/mini_al.h:7106:70: warning: passing argument 1 of 'mal_channel_map_to_channel_mask__opensl' discards 'const' qualifier from pointer target type
     pFormat->channelMask   = mal_channel_map_to_channel_mask__opensl(pConfig->channelMap, pFormat->numChannels);
                                                                      ^
In file included from external/mini_al.c:6:0:
external/mini_al.h:6748:17: note: expected 'mal_uint8 *' but argument is of type 'const mal_channel *'
 static SLuint32 mal_channel_map_to_channel_mask__opensl(mal_uint8 channelMap[MAL_MAX_CHANNELS], mal_uint32 channels)
                 ^
In file included from external/mini_al.c:6:0:
external/mini_al.h: In function 'mal_device_init':
external/mini_al.h:8738:10: warning: cast from pointer to integer of different size [-Wpointer-to-int-cast]
     if (((mal_uint64)pDevice % sizeof(pDevice)) != 0) {
          ^
external/mini_al.h: In function 'mal_src_read_frames_ex':
external/mini_al.h:9357:23: warning: variable 'algorithm' set but not used [-Wunused-but-set-variable]
     mal_src_algorithm algorithm = pSRC->config.algorithm;
                       ^
In file included from external/mini_al.c:6:0:
external/mini_al.c: At top level:
external/mini_al.h:1642:12: warning: 'mal_strcpy_s' defined but not used [-Wunused-function]
 static int mal_strcpy_s(char* dst, size_t dstSizeInBytes, const char* src)
            ^
external/mini_al.h:1701:12: warning: 'mal_strcat_s' defined but not used [-Wunused-function]
 static int mal_strcat_s(char* dst, size_t dstSizeInBytes, const char* src)
            ^
external/mini_al.h:1741:12: warning: 'mal_itoa_s' defined but not used [-Wunused-function]
 static int mal_itoa_s(int value, char* dst, size_t dstSizeInBytes, int radix)
            ^
external/mini_al.h:1807:12: warning: 'mal_strcmp' defined but not used [-Wunused-function]
 static int mal_strcmp(const char* str1, const char* str2)
            ^

PLATFORM_WEB (emscripten SDK 1.37.21)

Fails on compilation, tries to compile with OSS backend (Linux). Compilation output:

emcc -c external/mini_al.c -O1 -Wall -std=c99 -D_DEFAULT_SOURCE -fgnu89-inline -Wno-missing-braces -s USE_GLFW=3 -s ASSERTIONS=1 --profiling -I. -Iexternal -Iexternal/glfw/include -DPLATFORM_WEB 
In file included from external/mini_al.c:6:
In file included from external/mini_al.h:6343:
C:\emsdk\emscripten\1.37.21\system\include\libc\sys/soundcard.h:1:10: fatal error: 'linux/soundcard.h' file not found
#include <linux/soundcard.h>
         ^~~~~~~~~~~~~~~~~~~
1 error generated.
ERROR:root:compiler frontend failed to generate LLVM bitcode, halting

On PLATFORM_WEB it could probably be compiled for OpenAL backend (supported with no additional linkage). Is there any method to force a specific audio backend? Something like MAL_FORCE_OPENAL?

Also, is there any method to retrieve Audio Device information? It would be great to get some information similar to Graphic Device info:

INFO: Display device initialized successfully
INFO: Display size: 1920 x 1080
INFO: Render size: 800 x 450
INFO: Screen size: 800 x 450
INFO: Viewport offsets: 0, 0
INFO: GLAD: OpenGL extensions loaded successfully
INFO: OpenGL 3.3 Core profile supported
INFO: GPU: Vendor:   NVIDIA Corporation
INFO: GPU: Renderer: GeForce GT 730M/PCIe/SSE2
INFO: GPU: Version:  3.3.0 NVIDIA 376.54
INFO: GPU: GLSL:     3.30 NVIDIA via Cg compiler
INFO: Number of supported extensions: 326

[mackron]

1) Concerning the crash on Raspberry, I don't have a device to test on, but hopefully I can reproduce it on my desktop Linux machines and fix it. Do you have a stack trace? 2) I will look into the Android stuff, but I'm wondering if it's falling back to the Null backend for some reason... 3) I can already see what's wrong with the PLATFORM_WEB backend - it's not detecting Emscripten and is assuming it's a standard Unix platform and thus assumes OSS is the backend of choice. I'll get that fixed. 4) There is no way to force only a single backend at compile time except for MAL_NO_*, however you can force specific backends at run-time by specifying them when calling mal_context_init(). 5) Simple device information can be retrieved from the mal_device object:

device->format / device->internalFormat
device->channels / device->internalChannels
device->sampleRate / device->internalSampleRate

There is currently no way to retrieve the name of the device from a mal_device object. I'll consider this...

I'll some tracing to help with debugging these issues.

Some of those warnings have been fixed by the way - synchronize your local copy of the dr/mini_al branch :)

[raysan5]

Wow! Super fast response! Thanks for the answer! :D

Concerning the crash on Raspberry, I don't have a device to test on, but hopefully I can reproduce it on my desktop Linux machines and fix it. Do you have a stack trace?

Here the backtrace I got with GDB:

GNU gdb (Raspbian 7.12-6) 7.12.0.20161007-git
Copyright (C) 2016 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.  Type "show copying"
and "show warranty" for details.
This GDB was configured as "arm-linux-gnueabihf".
Type "show configuration" for configuration details.
For bug reporting instructions, please see:
<http://www.gnu.org/software/gdb/bugs/>.
Find the GDB manual and other documentation resources online at:
<http://www.gnu.org/software/gdb/documentation/>.
For help, type "help".
Type "apropos word" to search for commands related to "word"...
Reading symbols from ./audio_music_stream...done.
(gdb) run
Starting program: /home/pi/raylib/examples/audio/audio_music_stream
[Thread debugging using libthread_db enabled]
Using host libthread_db library "/lib/arm-linux-gnueabihf/libthread_db.so.1".
INFO: Initializing raylib (v1.8.0)
[New Thread 0x76cdd470 (LWP 649)]
[New Thread 0x762ff470 (LWP 650)]
[New Thread 0x75aff470 (LWP 651)]
[New Thread 0x752ff470 (LWP 652)]
INFO: UPSCALING: Required screen size: 800 x 450 -> Display size: 1920 x 1080
INFO: Display device initialized successfully
INFO: Display size: 1920 x 1080
INFO: Render size: 800 x 450
INFO: Screen size: 800 x 450
INFO: Viewport offsets: 0, 0
INFO: GPU: Vendor:   Broadcom
INFO: GPU: Renderer: VideoCore IV HW
INFO: GPU: Version:  OpenGL ES 2.0
INFO: GPU: GLSL:     OpenGL ES GLSL ES 1.00
INFO: Number of supported extensions: 13
WARNING: [EXTENSION] VAO extension not found, VAO usage not supported
INFO: [EXTENSION] NPOT textures extension detected, full NPOT textures supported
INFO: [EXTENSION] ETC1 compressed textures supported
INFO: [TEX ID 1] Texture created successfully (1x1)
INFO: [TEX ID 1] Base white texture loaded successfully
INFO: [SHDR ID 1] Shader compiled successfully
INFO: [SHDR ID 2] Shader compiled successfully
INFO: [SHDR ID 3] Shader program loaded successfully
INFO: [SHDR ID 3] Default shader loaded successfully
INFO: [CPU] Default buffers initialized successfully (lines, triangles, quads)
INFO: [VBO ID 1][VBO ID 3] Default buffers VBOs initialized successfully (lines)
INFO: [VBO ID 5][VBO ID 6] Default buffers VBOs initialized successfully (triangles)
INFO: [VBO ID 7][VBO ID 8][VBO ID 9][VBO ID 10] Default buffers VBOs initialized successfully (quads)
INFO: OpenGL default states initialized successfully
INFO: [TEX ID 2] Texture created successfully (128x128)
INFO: [TEX ID 2] Default font loaded successfully
[New Thread 0x74aff470 (LWP 653)]
INFO: Mouse device initialized successfully
WARNING: Touch device could not be opened, no touchscreen available
WARNING: Could not change keyboard mode (SSH keyboard?)
WARNING: Gamepad device could not be opened, no gamepad available
[New Thread 0x74212470 (LWP 654)]
INFO: [AUD ID 0] Audio stream loaded successfully (44100 Hz, 16 bit, Mono)
INFO: Target time per frame: 16.667 milliseconds

Thread 7 "audio_music_str" received signal SIGSEGV, Segmentation fault.
[Switching to Thread 0x74212470 (LWP 654)]
0x0004b8d4 in OnSendAudioDataToDevice (pDevice=0xa94f8 <device>, frameCount=552, pFramesOut=0x11e098) at audio.c:471
471                             MixFrames(framesOut, framesIn, framesJustRead, internalData->volume);

[mackron]

OK, I've pushed some changes which includes some logging of some device information. Are you able to update to this new version and show your output for the Raspberry and Android builds?

Regarding Emscripten, unfortunately I don't think it's going to work because mini_al is an asynchronous API and uses a background thread, which if my understanding is correct, Emscripten does not support. I also use dlopen/dlsym which I understand is also not supported (though I can easily enough work around that by linking at compile time for Emscripten builds). I'm not sure what to do with Emscripten...

Still working on the Rasberry and Android issues. I may need some help with those ones, though...

Another question: What model of Raspberry Pi do you have?

UPDATE: I've pushed a potential fix for the Android build. It was trying to dynamically link against pthread at run-time which fails. Tested against API level 16 on an emulator. Could you try that one again?

Regarding Emscripten, I'm going to do some research into an SDL backend and see how that works.

UPDATE 2: I've pushed a very experimental change that adds an SDL backend in an attempt to get Emscripten working. I've not had a chance to test and debug it properly, but could you give that version a go? You'll need to do -s USE_SDL=2 when compiling.

[raysan5]

Hi @mackron, thank you very much for the update!

Don't know if I understand it correctly... about emscripten, it support pthreads, actually, physic examples run on web using multiple threads.

Current OpenAL Soft backend works ok in web. Would it be possible to just force OpenAL mini_al backend on emscripten? emscripten supports OpenAL out of the box without any special linking flags or similar (https://github.com/kripken/emscripten/issues/666).

About Android and Raspberry Pi, you have all my help, I can not test it this weekend but on Monday morning I can work on that.

I'm usually testing on RPi 1 B+ with Raspbian Jessie and RPi 3 with Raspbian Stretch.

I think using emscripten SDL backend implies linking agains all SDL library, that's a bit overloading... Actually, raylib is kind of SDL alternative.

[mackron]

When researching I saw that link, and in particular, I was looking at this:

By default, support for pthreads is not enabled, since the specification is still in a prototyping stage.

mini_al should Just Work, and not having pthreads enabled by default (and working reliably) makes things too hard and too annoying.

I think using emscripten SDL backend implies linking agains all SDL library, that's a bit overloading... Actually, raylib is kind of SDL alternative.

Nope. Take a look a look at the latest update on dr/mini_al :-) I can get a slightly modified version of my simple_playback example (just to get it working with Emscripten) working with this:

emcc ../simple_playback.c -o simple_playback.html

The necessary packages come with Emscripten and there's no need for any special compile time flags.

On non-Emscripten platforms the SDL backend requires no linking and no headers. It Just Works without any need for installing any SDL development packages. It's also the lowest priority backend so it won't be picked by default in most cases (only on Emscripten). It links to SDL2.dll at run-time and if it doesn't find it, silently ignores SDL and tries the next backend.

[raysan5]

By default, support for pthreads is not enabled, since the specification is still in a prototyping stage.

raylib physics examples work with multithreading with no extra parameter or enabling anything in particular, just linking with -lpthreads. Neither -s USE_PTHREADS=1 is used on compilation despite being available.

The necessary packages come with Emscripten and there's no need for any special compile time flags.

Yes, emscripten comes with a bunch of libraries already compiled to asm.js, in most cases there is no need to link explicitly with those libraries, emscripten detects automatically its use and links, that happens for library_openal.js and also for library_sdl.js. So I proposed just falling to OpenAL backend, it should work out-of-the-box.

Tomorrow I'll take a closer look to all platforms implementations (Android, RPI, HTML5).

[mackron]

How is browser support for pthreads, do you know? It makes me nervous to have the Emscripten build be completely dependent on a system where the official documentation says it's still in the prototyping stage.

In my mind it makes so much more sense to use SDL because it's backend is way simpler than OpenAL and it avoids the threading issue entirely. It doesn't add any complexity to the build system either. I just don't see the downside... I might try pthreads+OpenAL at some point as an experiment, but I doubt it'll be as good as the SDL solution.

Also, I ordered a Raspberry Pi 3b :-)

[raysan5]

Hi @mackron, just tested current mini_al build on RaspberryPi 3, Android and HTML5.

On Raspberry Pi 3 got same result as before, here it's the output:

INFO: Initializing raylib (v1.8.0)
[New Thread 0x76cdd470 (LWP 913)]
[New Thread 0x762ff470 (LWP 914)]
[New Thread 0x75aff470 (LWP 915)]
[New Thread 0x752ff470 (LWP 916)]
INFO: UPSCALING: Required screen size: 800 x 450 -> Display size: 1920 x 1080
INFO: Display device initialized successfully
INFO: Display size: 1920 x 1080
INFO: Render size: 800 x 450
INFO: Screen size: 800 x 450
INFO: Viewport offsets: 0, 0
INFO: GPU: Vendor:   Broadcom
INFO: GPU: Renderer: VideoCore IV HW
INFO: GPU: Version:  OpenGL ES 2.0
INFO: GPU: GLSL:     OpenGL ES GLSL ES 1.00
INFO: Number of supported extensions: 13
WARNING: [EXTENSION] VAO extension not found, VAO usage not supported
INFO: [EXTENSION] NPOT textures extension detected, full NPOT textures supported
INFO: [EXTENSION] ETC1 compressed textures supported
INFO: [TEX ID 1] Texture created successfully (1x1)
INFO: [TEX ID 1] Base white texture loaded successfully
INFO: [SHDR ID 1] Shader compiled successfully
INFO: [SHDR ID 2] Shader compiled successfully
INFO: [SHDR ID 3] Shader program loaded successfully
INFO: [SHDR ID 3] Default shader loaded successfully
INFO: [CPU] Default buffers initialized successfully (lines, triangles, quads)
INFO: [VBO ID 1][VBO ID 3] Default buffers VBOs initialized successfully (lines)
INFO: [VBO ID 5][VBO ID 6] Default buffers VBOs initialized successfully (triangles)
INFO: [VBO ID 7][VBO ID 8][VBO ID 9][VBO ID 10] Default buffers VBOs initialized successfully (quads)
INFO: OpenGL default states initialized successfully
INFO: [TEX ID 2] Texture created successfully (128x128)
INFO: [TEX ID 2] Default font loaded successfully
[New Thread 0x74aff470 (LWP 917)]
INFO: Mouse device initialized successfully
WARNING: Touch device could not be opened, no touchscreen available
WARNING: Could not change keyboard mode (SSH keyboard?)
WARNING: Gamepad device could not be opened, no gamepad available
[New Thread 0x74212470 (LWP 918)]
INFO: Audio device initialized successfully: Default Playback Device
INFO: Audio backend: ALSA
INFO: Audio format: 32-bit IEEE Floating Point
INFO: Audio channels: 2
INFO: Audio sample rate: 44100
INFO: [AUD ID 0] Audio stream loaded successfully (44100 Hz, 16 bit, Mono)
INFO: Target time per frame: 16.667 milliseconds

Thread 7 "audio_music_str" received signal SIGSEGV, Segmentation fault.
[Switching to Thread 0x74212470 (LWP 918)]
0x0004b528 in OnSendAudioDataToDevice (pDevice=0xad9b0 <device>,
    frameCount=552, pFramesOut=0x122098) at audio.c:354
354                             MixFrames(framesOut, framesIn, framesJustRead, audioBuffer->volume);

On Android device just got no audio, actually, it fails even before showing the audio devide information, adding some TraceLog() messages, code fails at this point:

    // Keep the device running the whole time. May want to consider doing something a bit smarter and only have the device running
    // while there's at least one sound being played.
    result = mal_device_start(&device);
    if (result != MAL_SUCCESS)
    {
        TraceLog(LOG_WARNING, "[MINI_AL] Could not start audio device");
        mal_device_uninit(&device);
        mal_context_uninit(&context);
        return;
    }

On HTML5 now it compiles perfectly but when running I got the following Javascript output:

Initializing SDL audio threw an exception: "Invalid SDL audio format 33056!"! Continuing without audio.
ERROR: Failed to open SDL device.

About your questions:

How is browser support for pthreads, do you know? It makes me nervous to have the Emscripten build be completely dependent on a system where the official documentation says it's still in the prototyping stage.

Actually, I don't know exactly how it works but here I found some related information: https://github.com/kripken/emscripten/pull/5533

Checking your current implementation for SDL backend, I think it could be very similar with OpenAL backend, just MAL_NO_RUNTIME_LINKING with OpenAL and include required AL headers (or the functions definitions). It should be enough; actually that's how it currently works on raylib with emscripten and OpenAL backend. I can try to implement it myself if you want. :)

[mackron]

So the Android build is failing when it tries to start the device? Interesting... Are you trying that on real hardware or an emulator? If an emulator, can you give details so I can try the same one?

The HTML5 issue is due to me being stupid - I have an SDL 1.2 fallback (which is the default for Emscripten builds) in case SDL 2 isn't supported, but 1.2 doesn't support floating point. Will get that fixed up for you so you can try again.

Regarding the OpenAL stuff, if you want to do that, check for MAL_NO_RUNTIME_LINKING, and if defined, just #include the standard headers rather than inlining the function declarations. The reason is that it's most likely the compiler will have the AL headers if they are linking at compile time. This makes inlining the function declarations unnecessary, and in turn reduces maintenance costs for mini_al.

UPDATE: I just pushed a potential fix for the Emscripten build to dr/mini_al for you to try. Will look at the Raspberry and Android stuff later... I've tested the Android build on emulators, but I don't have an easy way to test on real hardware... Do you have a log I can look at for the Android build?

[raysan5]

Hi @mackron! Wow, what a fast response! :D

Are you trying that on real hardware or an emulator?

I'm trying it on a device, a Motorola MotoG (XT1032) with Android 5.0.

The HTML5 issue is due to me being stupid

Hehehe, no worries, it happens to me all the time! :P I'll try it and let you know... I will also try to update the OpenAL backend with MAL_NO_RUNTIME_LINKING when using emscripten.

Do you have a log I can look at for the Android build?

I can extract the full running log but unfortunately there is not much info about audio there... in any case, I'll port it here ASAP.

[raysan5]

I just pushed a potential fix for the Emscripten build to dr/mini_al for you to try.

Tried it and got this message (included by me on audio device start):

WARNING: [MINI_AL] Could not start audio device
SDL_QuitSubSystem called (and ignored)

Here the Android logcat output showing warnings and errors:

C:/android-sdk/platform-tools/adb logcat *:W
--------- beginning of system
--------- beginning of main
W/Quickoffice(15611): Cleanup of storage: done
W/ActivityManager(  834): No permission grants found for com.quickoffice.android
W/Quickoffice(15611): Could not load clipboard.
E/Finsky  (15382): [1] com.google.android.finsky.wear.bk.a(3): onConnectionFailed: ConnectionResult{statusCode=API_UNAVAILABLE, resolution=null, message=null}
W/Finsky  (15382): [1] com.google.android.finsky.wear.al.run(9): Dropping command=auto_install due to Gms not connected
W/Finsky  (15382): [1] com.google.android.finsky.wear.al.run(9): Dropping command=send_installed_apps due to Gms not connected
W/Quickoffice(15611): Could not store clipboard.
E/WifiStateMachine(  834): startDelayedScan send -> 5455 milli 10000
E/WifiStateMachine(  834): [1,511,205,277,923 ms] noteScanstart no scan source uid -2
E/WifiStateMachine(  834): [1,511,205,278,338 ms] noteScanEnd no scan source onTime=0
E/WifiStateMachine(  834): wifi setScanResults statecom.android.server.wifi.WifiStateMachine$DisconnectedState@26a50cdf sup_state=SCANNING debouncing=false mConnectionRequests=1 selection=<none>
E/NetlinkEvent(  271): NetlinkEvent::FindParam(): Parameter 'TIME_NS' not found
E/FastThread(  276): did not receive expected priority boost
E/Finsky  (15382): [43810] com.google.android.finsky.af.c.a(14): Unable to build selector: /storage/emulated/0/Download/marketenvs.csv: open failed: ENOENT (No such file or directory)
W/Settings(  834): Setting tether_dun_apn has moved from android.provider.Settings.Secure to android.provider.Settings.Global.
W/linker  (15669): libmain.so: unused DT entry: type 0x6ffffffe arg 0xa428
W/linker  (15669): libmain.so: unused DT entry: type 0x6fffffff arg 0x3
W/raylib  (15669): WARNING: Window set to landscape mode
W/IInputConnectionWrapper( 1580): showStatusIcon on inactive InputConnection
E/WindowManager(  834): Performed 6 layouts in a row. Skipping
W/AudioTrack(15669): AUDIO_OUTPUT_FLAG_FAST denied by client
W/raylib  (15669): WARNING: [MINI_AL] Could not start audio device
W/AudioFlinger(  276): removeEffect_l() 0xb5859008 cannot promote chain for effect 0xb8464ae8
W/AudioPolicyManager(  276): unregisterEffect() unknown effect ID 4165
E/FastThread(  276): did not receive expected priority boost
W/ResourceType( 1308): No package identifier when getting value for resource number 0x00000000
W/PackageManager( 1308): Failure retrieving resources for com.raylib.rgame: Resource ID #0x0
W/ResourceType( 1308): No package identifier when getting value for resource number 0x00000000
W/PackageManager( 1308): Failure retrieving resources for com.raylib.rgame: Resource ID #0x0
W/Settings(  834): Setting tether_dun_apn has moved from android.provider.Settings.Secure to android.provider.Settings.Global.
W/IInputConnectionWrapper( 1308): showStatusIcon on inactive InputConnection
E/WifiStateMachine(  834): startDelayedScan send -> 5456 milli 10000
E/WifiStateMachine(  834): [1,511,205,287,925 ms] noteScanstart no scan source uid -2
E/WifiStateMachine(  834): [1,511,205,288,290 ms] noteScanEnd no scan source onTime=0
E/WifiStateMachine(  834): wifi setScanResults statecom.android.server.wifi.WifiStateMachine$DisconnectedState@26a50cdf sup_state=SCANNING debouncing=false mConnectionRequests=1 selection=<none>
W/OpenGLRenderer( 1580): Incorrectly called buildLayer on View: ShortcutAndWidgetContainer, destroying layer...

[mackron]

Thanks for that! I've got a fix in now for both Android and SDL. The problem was that mal_device_start() was simply returning the wrong value for those backends (they run on a slightly different code path to the other backends and I didn't notice till now). Evidently I need to update my examples and tests...

Try that one out and see how you go!

[raysan5]

Hi @mackron! Tested new changes! Unfortunately, not working yet... it crashes on Android and HTML5.

Android logcat output (relevant parts):

...
I/raylib  ( 6878): INFO: APP_CMD_GAINED_FOCUS
W/AudioTrack( 6878): AUDIO_OUTPUT_FLAG_FAST denied by client
I/raylib  ( 6878): INFO: Audio device initialized successfully: Default Playback Device
I/raylib  ( 6878): INFO: Audio backend: OpenSL|ES
I/raylib  ( 6878): INFO: Audio format: 32-bit IEEE Floating Point
I/raylib  ( 6878): INFO: Audio channels: 2
I/raylib  ( 6878): INFO: Audio sample rate: 44100
I/raylib  ( 6878): INFO: [resources/mecha.png] Image loaded successfully (128x128)
I/raylib  ( 6878): INFO: [TEX ID 3] Texture created successfully (128x128)
I/raylib  ( 6878): INFO: Image file loaded correctly as SpriteFont
I/raylib  ( 6878): INFO: [AUD ID 0] Audio stream loaded successfully (44100 Hz, 16 bit, Stereo)
I/raylib  ( 6878): INFO: [resources/coin.wav] WAV file loaded successfully (22050 Hz, 16 bit, Mono)
I/raylib  ( 6878): INFO: Unloaded wave data from RAM
I/raylib  ( 6878): INFO: [resources/raylib_logo.png] Image loaded successfully (256x256)
I/raylib  ( 6878): INFO: [TEX ID 4] Texture created successfully (256x256)
I/raylib  ( 6878): INFO: Target time per frame: 16.667 milliseconds
--------- beginning of crash
F/libc    ( 6878): stack corruption detected
F/libc    ( 6878): Fatal signal 6 (SIGABRT), code -6 in tid 6911 (AudioTrack)
W/ActivityManager(  834): Force removing ActivityRecord{2ac69b05 u0 com.raylib.rgame/.NativeLoader t1089}: app died, no saved state
W/AudioPolicyManager(  276): unregisterEffect() unknown effect ID 4794
W/AudioFlinger(  276): session id 4793 not found for pid 6878
W/AudioFlinger(  276): removeEffect_l() 0xb5859008 cannot promote chain for effect 0xb8466800
W/AudioPolicyManager(  276): unregisterEffect() unknown effect ID 4794
E/AppsCustomizePagedView( 1580): Widget ComponentInfo{com.google.android.apps.docs/com.google.android.apps.docs.widget.CakemixAppWidgetProvider} can not fit on this device (588, 120)
E/AppsCustomizePagedView( 1580): Widget ComponentInfo{com.etrade.mobilepro.activity/com.etrade.mobilepro.activity.ETAppWidgetProvider} can not fit on this device (588, 292)
W/Settings(  834): Setting tether_dun_apn has moved from android.provider.Settings.Secure to android.provider.Settings.Global.
W/Launcher.Model( 1580): LoaderTask running with no launcher (onlyBindAllApps)
W/ResourceType( 1308): No package identifier when getting value for resource number 0x00000000
W/PackageManager( 1308): Failure retrieving resources for com.raylib.rgame: Resource ID #0x0
W/ProcessCpuTracker(  834): Skipping unknown process pid 6923
W/ProcessCpuTracker(  834): Skipping unknown process pid 6926
W/ProcessCpuTracker(  834): Skipping unknown process pid 6934
W/ProcessCpuTracker(  834): Skipping unknown process pid 6935
W/InputMethodManagerService(  834): Got RemoteException sending setActive(false) notification to pid 6878 uid 10274

HTML5 console output (relevant parts):

...
INFO: Audio device initialized successfully: Default Playback Device
INFO: Audio backend: Unknown
INFO: Audio format: 32-bit IEEE Floating Point
INFO: Audio channels: 2
INFO: Audio sample rate: 44100
INFO: [AUD ID 0] Audio stream loaded successfully (44100 Hz, 16 bit, Mono)
Invalid function pointer '0' called with signature 'iiiii'. Perhaps this is an invalid value (e.g. caused by calling a virtual method on a NULL pointer)? Or calling a function with an incorrect type, which will fail? (it is worth building your source files with -Werror (warnings are errors), as warnings can indicate undefined behavior which can cause this)
printErr @ audio_music_stream.html:191
nullFunc_iiiii @ audio_music_stream.js:11757
b131 @ audio_music_stream.js:78416
_mal_dsp_read_frames_ex @ audio_music_stream.js:42369
_OnSendAudioDataToDevice @ audio_music_stream.js:38334
_mal_device__on_read_from_client @ audio_music_stream.js:40780
_mal_dsp_read_frames_ex @ audio_music_stream.js:42369
_mal_dsp_read_frames @ audio_music_stream.js:42236
_mal_device__read_frames_from_client @ audio_music_stream.js:42088
_mal_audio_callback__sdl @ audio_music_stream.js:41978
dynCall_viii @ audio_music_stream.js:77559
SDL_queueNewAudioData @ audio_music_stream.js:11201
Browser_mainLoop_runner @ audio_music_stream.js:2212
requestAnimationFrame (async)
requestAnimationFrame @ audio_music_stream.js:2584
Browser_mainLoop_scheduler_rAF @ audio_music_stream.js:2101
_emscripten_set_main_loop @ audio_music_stream.js:2221
_main @ audio_music_stream.js:12479
asm._main @ audio_music_stream.js:78616
callMain @ audio_music_stream.js:78888
doRun @ audio_music_stream.js:78952
(anonymous) @ audio_music_stream.js:78963
setTimeout (async)
run @ audio_music_stream.js:78959
runCaller @ audio_music_stream.js:78858
removeRunDependency @ audio_music_stream.js:1768
processPackageData @ audio_music_stream.js:161
(anonymous) @ audio_music_stream.js:93
xhr.onload @ audio_music_stream.js:76
XMLHttpRequest.send (async)
fetchRemotePackage @ audio_music_stream.js:81
loadPackage @ audio_music_stream.js:91
(anonymous) @ audio_music_stream.js:185
(anonymous) @ audio_music_stream.js:187
audio_music_stream.html:191 This pointer might make sense in another type signature: iiii: 0  iii: 0  ii: 0  i: 0  viiii: 0  viii: 0  viiiii: 0  vii: 0  vi: 0  viid: 0  viiiiii: 0  vid: 0  vdi: 0  vidd: 0  vd: 0  viiiiiii: 0  vdd: 0  viddd: 0  v: 0  viiiiiiii: 0  vidddd: 0  vdddd: 0  viiiiiiiii: 0  vdddddd: 0  
printErr @ audio_music_stream.html:191
nullFunc_iiiii @ audio_music_stream.js:11757
b131 @ audio_music_stream.js:78416
_mal_dsp_read_frames_ex @ audio_music_stream.js:42369
_OnSendAudioDataToDevice @ audio_music_stream.js:38334
_mal_device__on_read_from_client @ audio_music_stream.js:40780
_mal_dsp_read_frames_ex @ audio_music_stream.js:42369
_mal_dsp_read_frames @ audio_music_stream.js:42236
_mal_device__read_frames_from_client @ audio_music_stream.js:42088
_mal_audio_callback__sdl @ audio_music_stream.js:41978
dynCall_viii @ audio_music_stream.js:77559
SDL_queueNewAudioData @ audio_music_stream.js:11201
Browser_mainLoop_runner @ audio_music_stream.js:2212
requestAnimationFrame (async)
requestAnimationFrame @ audio_music_stream.js:2584
Browser_mainLoop_scheduler_rAF @ audio_music_stream.js:2101
_emscripten_set_main_loop @ audio_music_stream.js:2221
_main @ audio_music_stream.js:12479
asm._main @ audio_music_stream.js:78616
callMain @ audio_music_stream.js:78888
doRun @ audio_music_stream.js:78952
(anonymous) @ audio_music_stream.js:78963
setTimeout (async)
run @ audio_music_stream.js:78959
runCaller @ audio_music_stream.js:78858
removeRunDependency @ audio_music_stream.js:1768
processPackageData @ audio_music_stream.js:161
(anonymous) @ audio_music_stream.js:93
xhr.onload @ audio_music_stream.js:76
XMLHttpRequest.send (async)
fetchRemotePackage @ audio_music_stream.js:81
loadPackage @ audio_music_stream.js:91
(anonymous) @ audio_music_stream.js:185
(anonymous) @ audio_music_stream.js:187
audio_music_stream.html:179 0
audio_music_stream.html:191 0
printErr @ audio_music_stream.html:191
abort @ audio_music_stream.js:79007
nullFunc_iiiii @ audio_music_stream.js:11757
b131 @ audio_music_stream.js:78416
_mal_dsp_read_frames_ex @ audio_music_stream.js:42369
_OnSendAudioDataToDevice @ audio_music_stream.js:38334
_mal_device__on_read_from_client @ audio_music_stream.js:40780
_mal_dsp_read_frames_ex @ audio_music_stream.js:42369
_mal_dsp_read_frames @ audio_music_stream.js:42236
_mal_device__read_frames_from_client @ audio_music_stream.js:42088
_mal_audio_callback__sdl @ audio_music_stream.js:41978
dynCall_viii @ audio_music_stream.js:77559
SDL_queueNewAudioData @ audio_music_stream.js:11201
Browser_mainLoop_runner @ audio_music_stream.js:2212
requestAnimationFrame (async)
requestAnimationFrame @ audio_music_stream.js:2584
Browser_mainLoop_scheduler_rAF @ audio_music_stream.js:2101
_emscripten_set_main_loop @ audio_music_stream.js:2221
_main @ audio_music_stream.js:12479
asm._main @ audio_music_stream.js:78616
callMain @ audio_music_stream.js:78888
doRun @ audio_music_stream.js:78952
(anonymous) @ audio_music_stream.js:78963
setTimeout (async)
run @ audio_music_stream.js:78959
runCaller @ audio_music_stream.js:78858
removeRunDependency @ audio_music_stream.js:1768
processPackageData @ audio_music_stream.js:161
(anonymous) @ audio_music_stream.js:93
xhr.onload @ audio_music_stream.js:76
XMLHttpRequest.send (async)
fetchRemotePackage @ audio_music_stream.js:81
loadPackage @ audio_music_stream.js:91
(anonymous) @ audio_music_stream.js:185
(anonymous) @ audio_music_stream.js:187
audio_music_stream.js:79024 Uncaught abort(0) at Error
    at jsStackTrace (http://localhost:8080/audio_music_stream.js:1313:13)
    at stackTrace (http://localhost:8080/audio_music_stream.js:1330:12)
    at abort (http://localhost:8080/audio_music_stream.js:79018:44)
    at nullFunc_iiiii (http://localhost:8080/audio_music_stream.js:11757:1364)
    at Array.b131 (http://localhost:8080/audio_music_stream.js:78416:43)
    at _mal_dsp_read_frames_ex (http://localhost:8080/audio_music_stream.js:42369:40)
    at Array._OnSendAudioDataToDevice (http://localhost:8080/audio_music_stream.js:38334:15)
    at Array._mal_device__on_read_from_client (http://localhost:8080/audio_music_stream.js:40780:39)
    at _mal_dsp_read_frames_ex (http://localhost:8080/audio_music_stream.js:42369:40)
    at _mal_dsp_read_frames (http://localhost:8080/audio_music_stream.js:42236:8)
abort @ audio_music_stream.js:79024
nullFunc_iiiii @ audio_music_stream.js:11757
b131 @ audio_music_stream.js:78416
_mal_dsp_read_frames_ex @ audio_music_stream.js:42369
_OnSendAudioDataToDevice @ audio_music_stream.js:38334
_mal_device__on_read_from_client @ audio_music_stream.js:40780
_mal_dsp_read_frames_ex @ audio_music_stream.js:42369
_mal_dsp_read_frames @ audio_music_stream.js:42236
_mal_device__read_frames_from_client @ audio_music_stream.js:42088
_mal_audio_callback__sdl @ audio_music_stream.js:41978
dynCall_viii @ audio_music_stream.js:77559
SDL_queueNewAudioData @ audio_music_stream.js:11201
Browser_mainLoop_runner @ audio_music_stream.js:2212
requestAnimationFrame (async)
requestAnimationFrame @ audio_music_stream.js:2584
Browser_mainLoop_scheduler_rAF @ audio_music_stream.js:2101
_emscripten_set_main_loop @ audio_music_stream.js:2221
_main @ audio_music_stream.js:12479
asm._main @ audio_music_stream.js:78616
callMain @ audio_music_stream.js:78888
doRun @ audio_music_stream.js:78952
(anonymous) @ audio_music_stream.js:78963
setTimeout (async)
run @ audio_music_stream.js:78959
runCaller @ audio_music_stream.js:78858
removeRunDependency @ audio_music_stream.js:1768
processPackageData @ audio_music_stream.js:161
(anonymous) @ audio_music_stream.js:93
xhr.onload @ audio_music_stream.js:76
XMLHttpRequest.send (async)
fetchRemotePackage @ audio_music_stream.js:81
loadPackage @ audio_music_stream.js:91
(anonymous) @ audio_music_stream.js:185
(anonymous) @ audio_music_stream.js:187

[mackron]

Thanks! How frustrating... Potential fix is in. Assuming that was it, that was actually a bug in the new mixing code I did for raylib, not mini_al.

Once you get a chance to test that out and you're happy with the results I'll bump the version of mini_al and do a pull request.

[raysan5]

Hey @mackron! GREAT NEWS! It works! :D

On Android platform it works perfectly.

On HTML5 platform it works ok but there seems to be some problem with buffer initialization and there is some delay in sound... you can check a music example (weird init behaviour) and sound example (long delay before playing).

On Raspberry Pi 3 platform it works but sound seems to be reproduced at wrong sample-rate, extremely slow playing.

Many thanks for your patience and your hardwork on this issue!

[mackron]

Well at least Android is figured out...

I'm not sure what's causing the issues with Emscripten and Raspberry Pi. I'll need to look at that in more detail later. Do you hear any glitching with Raspberry Pi, or is it just slow like a pitch shift?

[raysan5]

Tried Raspberry Pi more carefully and I'm afraid I can not explain in detail how it sounds... it seems kind of pitch shift with some audio saturation noise... but what I did was measuring RPI music playing time in comparison to original song played in Windows version and, in Raspberry Pi, it took approximately twice the time to play the same song.

EDIT: Just in case it could be useful:

Output Windows:

INFO: Audio device initialized successfully: Default Playback Device
INFO: Audio backend: WinMM
INFO: Audio format: 32-bit IEEE Floating Point
INFO: Audio channels: 2
INFO: Audio sample rate: 44100
INFO: [AUD ID 0] Audio stream loaded successfully (44100 Hz, 16 bit, Mono)

Output Raspberry Pi:

INFO: Audio device initialized successfully: Default Playback Device
INFO: Audio backend: ALSA
INFO: Audio format: 32-bit IEEE Floating Point
INFO: Audio channels: 2
INFO: Audio sample rate: 44100
INFO: [AUD ID 0] Audio stream loaded successfully (44100 Hz, 16 bit, Mono)

[mackron]

OK, I've pushed some potential fixes for RPI and SDL, but there's a few things I need you to do for me.

Raspberry Pi Unfortunately my RPI has horrible interference with the analogue output to the point that I can barely hear anything which makes it very hard for me to tell if my fixes are working. I have added a little piece of code at line 404 in audio.c with some RPI specific device configuration settings:

• Could you first try the settings I've already got in place?
• Could you try toggling alsa.noMMap mode between true and false and see if that makes any difference?
• Could you also try commenting out the bufferSizeInFrames = 2048 and see if that makes a difference? (Not explicitly setting this property makes it fall back to defaults.)
• Could you try setting alsa.noMMap = MAL_FALSE and changing DEVICE_SAMPLE_RATE to 48000 (I had a bug recently with 44100 on ALSA backends and I'm wondering if it's related)

When I use a USB output device everything works perfectly fine for me on RPI. It's only the built-in analogue output that has issues for me (though, I can't test HDMI output because I don't have a device to test with).

Emscripten I think the long delay is a result of an excessively large internal buffer which thus resulted in excessive latency. I've put a potential fix in place, but if that doesn't work, could you comment out line 8401 #define MAL_USE_SDL_1 in mini_al.h and pass -s USE_SDL=2 when compiling and see if that makes a difference?

Regarding the issue with what sounds like a buffer initialization glitch, assuming we're talking about the same thing, I think that issue is not so much a bug with the audio system, but rather the way in which the music stream is updated. It looks to me in my testing that the first call to EndFrame() takes a very long time which causes a delay for the next call to UpdateMusicStream() - long enough that the audio system is starved of new data. I think if you were to move UpdateMusicStream() to after the rendering section it would work fine, though I've not tested.

Could you also show your output again like you did in your previous message? I've update it with some more detailed information.

[raysan5]

Hi @mackron, thanks for the quick update!

Not sure if I could look at this during the weekend, I've got my RPIs at work... maybe I could try the Emscripten version... in any case, I'll let you know my results ASAP.

[raysan5]

I'm testing everything right now, just a detail:

When I use a USB output device everything works perfectly fine for me on RPI. It's only the built-in analogue output that has issues for me (though, I can't test HDMI output because I don't have a device to test with).

I use HDMI output by default, I can also try analogue output...

More info in a while!

[raysan5]

Ok, everything tested, here my results:

Raspberry Pi 3

First I tried all proposed solutions with no luck, also tried analogue jack output vs HDMI output, jack output has some interferences/noise, HDMI output was clear.

After some tests, found the solution to get it working:

deviceConfig.bufferSizeInFrames = 32768;

That was the minimum value to get it working in par with Windows desktop.

Output info:

INFO: Audio device initialized successfully: Default Playback Device
INFO: Audio backend: ALSA
INFO: Audio format: 32-bit IEEE Floating Point
INFO: Audio channels: 2
INFO: Audio sample rate: 44100
INFO: [AUD ID 0] Audio stream loaded successfully (44100 Hz, 16 bit, Mono)

HTML5 (emscripten)

Tried modified code and the delay is way shorter but you can hear some glitches/noise on music and sounds.

After that tried with line commented and -s USE=SDL=2 and it works perfectly, no delay and no glitches/noise.

Output info:

INFO: Audio device initialized successfully: Default Playback Device
INFO: Audio backend: SDL
INFO: Audio format: 32-bit IEEE Floating Point
INFO: Audio channels: 2
INFO: Audio sample rate: 44100
INFO: [AUD ID 0] Audio stream loaded successfully (44100 Hz, 16 bit, Mono)

The only problem I find is that linking against full SDL2 generates a ~9MB file instead of the ~900KB file generated when using OpenAL Soft backend...

[mackron]

Thanks for your feedback on this!

A buffer size of 32768 is huge - what is the latency like with that? In any case, I'm suspecting it's just because the RPI has crappy built-in audio which isn't suited for low-latency applications. I'm just not sure how I could detect the RPI at run-time and automatically tune it. I really don't want applications to require a compile time macro like "PLATFORM_RPI" - I want mini_al to Just Work. If you have any ideas, let me know!

A 9MB output for HTML5 is no good - I'll look into that. Does emcc have a "-s" option like GCC? Does that do anything? What is the output size with "-O2"?

I will look into the SDL issues and try getting OpenAL working with Emscripten, but I'll be out of action on this project for the next few days so I don't know when I'll get a chance to get to it.

[r-lyeh-archived]

@mackron https://raspberrypi.stackexchange.com/questions/754/how-can-i-detect-that-im-compiling-for-raspberry-pi ?

ifdef __arm__ && stat(/opt/vc/include/bcm_host.h) >= 0 :)

[raysan5]

A buffer size of 32768 is huge - what is the latency like with that?

Yes, it's quite big... but actually latency is not that bad, when playing a sound there is a delay between key pressed and playing of around 0.5 seconds.

If you have any ideas, let me know!

Actually not many ideas but I remember a similar issue with OpenAL related to AUDIO_BUFFER_SIZE, not being big enough, just checked code usage of that parameter:

    // The size of a streaming buffer must be at least double the size of a period.
    unsigned int periodSize = device.bufferSizeInFrames / device.periods;
    unsigned int subBufferSize = AUDIO_BUFFER_SIZE;
    if (subBufferSize < periodSize) {
        subBufferSize = periodSize;
    }

I saw device.periods defaults to 2, so checking buffer for refill twice every buffer size... maybe issue is related with that, maybe check timming is not accurate... I got a similar issue on windows with minimum time resolution... not sure, just an idea...

A 9MB output for HTML5 is no good - I'll look into that. Does emcc have a "-s" option like GCC? Does that do anything? What is the output size with "-O2"?

There is no -s for strip on emcc, using -O2 reduces generated javascript filesize to ~4.5MB.

...I'll be out of action on this project for the next few days so I don't know when I'll get a chance to get to it.

No worries @mackron, I'm also quite busy lately on my current job...

EDIT: @r-lyeh, I think @mackron means not requiring any platform dependent check to get it working correctly, just get a generic solution.

[mackron]

I can't reproduce the crackling issue on my machine for SDL. Could you perhaps try playing around with that deviceConfig.bufferSizeInFrames trick and see if that helps at all? Just wondering if perhaps the buffer size is too small, thus resulting in glitches.

[mackron]

I've pushed a potential fix for RPI, but I was hoping you could test something out for me. I have some logic in mini_al where it scales the default buffer size depending on known audio devices (currently only the RPI audio devices are used). I've currently got this set the 32, but I'm not sure if that's over the top. Would you be able to play around with those numbers to see what values work for you? The lower the better... Said code looks like the following, at about line 5484.

struct
{
    const char* name;
    float scale;
} g_malDefaultBufferSizeScalesALSA[] = {
    {"bcm2835 IEC958/HDMI", 32},
    {"bcm2835 ALSA",        32}
};

"bcm2835 ALSA" is the important one.

[raysan5]

Hi @mackron, just tested new code. Tried with values: 32, 16, 8.

Using HDMI audio output. With 32 and 16 it sounds very similar to me, probably better with 16... despite some small glitch (maybe some interference), both sound good. With 8 value it seems to be more glitchy, like audio saturation... In all cases, when playing sounds, seems to be a small delay at the beggining (comparing to Windows playing) but is less than half second...

Sorry for not being able to provide more detailed information.

On loading, output info is similar in all the modes (just changes the Audio buffer size data):

INFO: Audio device initialized successfully: Default Playback Device
INFO: Audio backend: mini_al / ALSA
INFO: Audio format: 32-bit IEEE Floating Point -> 32-bit IEEE Floating Point
INFO: Audio channels: 2 -> 2
INFO: Audio sample rate: 44100 -> 44100
INFO: Audio buffer size: 17600

I know it's a bit difficult to track those small issues working remotely... but now it sounds pretty good to me, maybe we can integrate those changes and keep working from there?

[mackron]

Pull request submitted. Probably best to move this discussion to over there.

[raysan5]

Hi @mackron, do you think we can close this issue for now?

[mackron]

Yep, closing.