Gap between sound loop

[piotrros]

• cocos2d-x version: 3.16
• devices test on: Nexus 6P
• developing environments
  • NDK version: r13b
  • Xcode version: 9.0
  • VS version: N/A
  • browser type and version: N/A

Steps to Reproduce:

So, in my app, I have a music playing in a loop. It's 26 seconds long.

On iOS I'm using caf file format and it works fine. On Android, I tried using wav (2.3 MB) or ogg (217 KB). Tried different bitrates (ogg) and also mono/stereo, but it doesn't really matter.

Using SimpleAudioEngine it only loops first few seconds - I've found there's a fixed limit of file size. I've also tried using playBackgroundMusic instead of playEffect. Then I switched to AudioEngine and there's a 0.3-0.5s gap between loop.

This gap is on Nexus 6P (android 8.0), but it's working fine on Samsung Galaxy J3 2016 (android 5.1.1).

I've tried a different, shorter sound (5 seconds long), and on Nexus 6P it's looping without gaps.

I need this song play gapless :)

Here's the song in WAV:

http://www94.zippyshare.com/v/2rCqA7GK/file.html

and in OGG:

http://www56.zippyshare.com/v/H22xN5h7/file.html

[piotrros]

@mingoo can you take a look?

[dumganhar]

The gap is caused by OpenSLES internally, when the ogg file size is bigger than 128K, we use UrlAudioPlayer for playing audio. UrlAudioPlayer streams and mixes audios in system side which cost a lot. The implementation of UrlAudioPlayer is really simple, just some invocations of OpenSLES so I could confirm that it's the bug of Android.

For the further, we need to deprecate UrlAudioPlayer just use PcmAudioPlayer but before that we need to implement streaming for PcmAudioPlayer first.

For now, you could change the configuration of how to select UrlAudioPlayer or PcmAudioPlayer in AudioPlayerProvider.cpp.

static AudioFileIndicator __audioFileIndicator[] = {
        {"default", 128000}, // If we could not handle the audio format, return default value, the position should be first.
        {".wav",    1024000},
        {".ogg",    128000}, // Changes these values bigger than your background music size to ensure only PcmAudioPlayer work for all audios.
        {".mp3",    160000}
};

Changing configuration will make the preload waste more time, therefore it's better to preload background music in the loading scene. And you have to listen the preload callback while loading. Since AudioEngine::preload is asynchronous, you need to add a callback for listen preload finish event.

AudioEngine::preload("a.mp3", [](){
   // a.mp3 has been finished.
});

[piotrros]

So, in other words, it works like that:

1) when file is smaller than limit it's loading a whole file 2) limit is needed because the whole file is loaded and it can cause memory problems and/or long loading on bigger files. Instead, it should use streaming, but it's not implemented yet. 3) as a workaround, it's using UrlAudioPlayer for bigger files, but it has its downsides such as audio won't loop without gaps.

Now I understand why AudioEngine is still experimental. It needs more work, I guess :)

After changing the limit to 512k it works flawlessly.

I have to say this: audio on Android is hell :d

I'd like to help implementing streaming, but I don't know where to start. Also, probably it's not just Android, but other platforms too.

[dumganhar]

Other platforms like macOS, iOS, win32 which are using OpenAL for the backend of AudioEngine is support streaming. Streaming is needed to be implemented for Android only. Basically, we need to implement a partial audio decoder rather that whole file decoder. tremolo which is used for decoding ogg files and pvmp3dec for decoding mp3 files should support partial decoding since Android uses them internally. But AudioDecoderSLES.cpp which is used for decoding audios exclude ogg, mp3, wav, is hard to implement partial decoding. AudioDecoderSLES uses OpenSLES API for decoding. I have no idea how to make that work. Perhaps, we could only support ogg, mp3, wav for Android and remove AudioDecoderSLES.

[piotrros]

Seems hard, I don't think I have enough knowledge about audio to do this :( But this needs to be fixed. Why audio on Android has so many problems :( ?

[deguilardi]

Having the very same problem here

[deguilardi]

after reading this: https://issuetracker.google.com/issues/36931073

I concluded there is nothing that can be done but a workaround Then I applied this: http://discuss.cocos2d-x.org/t/background-music-loop-on-android/22519/4

And now everything works fine!

[halx99]

@dumganhar I think you can use OpenAL + mpg123 & ogg at android platform.

[deguilardi]

I have used the "two players" workaround for years now. After so much time, I think this could be an acceptable solution. The solution described on the forum topic above generates some issues. I solved them with this following final implementation on my Cocos2dxMusic.java:

private void createNextMediaPlayer (final String path) throws IllegalStateException {
        new Thread(new Runnable() {
            @Override
            public void run() {
                try {
                    mBackgroundMediaPlayer2 = createMediaPlayer(path);
                    mBackgroundMediaPlayer.setNextMediaPlayer(mBackgroundMediaPlayer2);
                    mBackgroundMediaPlayer.setOnCompletionListener(onCompletionListener);
                }
                catch (IllegalStateException e) {
                }
            }
        }).start();
    }
    private MediaPlayer.OnCompletionListener onCompletionListener = new MediaPlayer.OnCompletionListener () {
        @Override
        public void onCompletion(MediaPlayer mediaPlayer) {
            mediaPlayer.release();
            if(mBackgroundMediaPlayer2 == null){
                mBackgroundMediaPlayer2 = createMediaPlayer (mCurrentPath);
            }
            if(mBackgroundMediaPlayer2 != null) {
                mBackgroundMediaPlayer.release();
                mBackgroundMediaPlayer = mBackgroundMediaPlayer2;
                try {
                    createNextMediaPlayer(mCurrentPath);
                } catch (IllegalStateException e) {
                }
            }
        }
    };

[Xrysnow]

I'm using OpenAL on android and it seems work fine. And I use streaming method in playback to get gapless loop. My repo is here, hope can help someone.

[halx99]

Well, OpenAL is a good backend, we also use it for all platforns.

[minggo]

@Xrysnow @halx99 How it affect application size?

[Xrysnow]

@minggo The libopenal.so file I compiled is about 450KB, much smaller than cocos (about 18MB). So I think it won't affect much.

[minggo]

@Xrysnow thanks for the information. And how about the compatibility in different Android devices?

[halx99]

Hi, our game is published at tencent platform: https://hongjing.qq.com and so far, it seems it is very stable. may be, my repo is easy to merge to the official engine, if you have any plan, you can see: https://github.com/halx99/x-studio365/tree/master/cocos2d-x-patch