Erratic Playback Behavior for ConcatenatingAudioSource with HLS Streams without #EXT-X-ENDLIST tag

[liorshk]

Which API doesn't behave as documented, and how does it misbehave? The issue is observed with just_audio, specifically with ConcatenatingAudioSource and HlsAudioSource. The misbehaviors are:

1. After pausing the playback of the first item in an HLS audio stream, the playback erroneously advances to the next item (10-20 seconds later) and continues to do so (even though we are not doing anything and it's supposed to be in "pause" state)
2. Then, when switching between items (Skipping to the next item) in the stream, playback sometimes begins from a mid-segment of the item, rather than from the first segment.

Minimal reproduction project https://github.com/liorshk/just_audio_demo

To Reproduce (i.e. user steps, not code) Steps to reproduce the behavior:

1. Play an .m3u8 without a #EXT-X-ENDLIST.
2. Pause the playback.
3. Observe that the playback advances to the next item (Issue 1).
4. Switch to a different item in the stream.
5. Observe that playback sometimes starts from a mid-segment of the new item (Issue 2).

Error messages

None

Expected behavior Upon pausing, the playback should remain on the current item without progressing.
Playback should start from the first segment of any selected item in the HLS stream.

Screenshots

Desktop (please complete the following information): Not relevant for desktop

Smartphone (please complete the following information): Device: Android and IOS

Flutter SDK version

[✓] Flutter (Channel stable, 3.16.0, on macOS 14.1.1 23B81 darwin-x64)
[✓] Android toolchain - develop for Android devices (Android SDK version 34.0.0)
[✓] Xcode - develop for iOS and macOS (Xcode 15.0.1)
[✓] Chrome - develop for the web
[✓] Android Studio (version 2022.3)
[✓] VS Code (version 1.84.2)

Additional context When trying to put additional logs in just_audio library, it seems like the event that triggers the index change is

_playbackEventSubscription =
          platform.playbackEventMessageStream.listen((message) {

I am not sure why the current index changes (issue 1).
 Moreover, I don't know why when moving between items, it starts with different segments (m3u8 segments) and not the 1st segment.
In addition, it seems like there are seeking issues. For example, when I pause and then start again, it sometimes goes to the beginning of the streaming segment, rather than the second I stopped at.

M3u8 Example


#EXTM3U

#EXT-X-VERSION:3

#EXT-X-TARGETDURATION:9

#EXT-X-PLAYLIST-TYPE:EVENT

#EXTINF:8.616,
https://storage.googleapis.com/.._1.aac

#EXTINF:8.328,
https://storage.googleapis.com/.._2.aac

#EXTINF:5.016,
https://storage.googleapis.com/.._3.aac

#EXTINF:3.288,
https://storage.googleapis.com/.._4.aac

Each playlist item consists of different m3u8 files, and each file has multiple segments ("event stream")

[ryanheise]

Can you please edit your bug report to include an actual reproduction project that is ready for me to run?

[liorshk]

Hey @ryanheise , Thanks for the quick response. I cloned the example from your tutorials: https://github.com/suragch/audio_playlist_flutter_demo.git and changed _setInitialPlaylist from ConcatenatingAudioSource of AudioSource to ConcatenatingAudioSource of HlsAudioSource with .m3u8 playlists. As follows:

void _setInitialPlaylist() async {
    final stream1 = Uri.parse('https://storage.googleapis.com/[hidden_link_1].m3u8');
    final stream2 = Uri.parse('https://storage.googleapis.com/[hidden_link_2].m3u8');
    final stream3 = Uri.parse('https://storage.googleapis.com/[hidden_link_3].m3u8');
    _playlist = ConcatenatingAudioSource(children: [
      HlsAudioSource(stream1, tag: '1'),
      HlsAudioSource(stream2, tag: '2'),
      HlsAudioSource(stream3, tag: '3'),
    ]);
    await _audioPlayer.setAudioSource(_playlist);
  }

I created a sample repository with the actual links that I used for your convenience as well: Sample Repo Demonstrating the issue

This is a video of the behavior:

https://github.com/ryanheise/just_audio/assets/7127502/450f586b-c2e7-425b-a192-2b11319b03c1

As you can see, it's possible to reproduce it by following the steps below:

• Play the first item
• Pause the playback.
• Observe that the playback advances to the next item (at 00:16:00)
• Observe that playback sometimes starts from a mid-segment of the new item (also at 00:16:00)

[ryanheise]

Oh, apologies if I was unclear, you can check the original instructions on the bug report on how to create a minimal reproduction project. This is done by forking this repository and modifying the example, not by modifying a 3rd party tutorial. Then I can see the diffs of what you changed. Once you've checked the original instructions, please edit your bug report above according to the original instructions.

[liorshk]

@ryanheise Sure, here you go: Link to just_audio fork You can run it by running the example_playlist flutter run -t lib/example_playlist.dart

The same steps reproduce the issue as displayed in the previous message.

[liorshk]

@ryanheise Quick update - it seems like the issue is specifically with .m3u8 files without #EXT-X-ENDLIST at the end. This is standard for EXT-X-PLAYLIST-TYPE:EVENT but it doesn't work. It should work fine with .m3u8 files without the #EXT-X-ENDLIST tag. I updated the example to reflect both types of files.

[ryanheise]

Checking your code, I can see that it is not doing anything that would start just_audio's proxy server in which case just_audio's m3u8 parser is not involved here. So it would "seem" this behaviour would be the native behaviour of the operating system's player which still leaves open the mystery of what's happening.

Are you indicating in your report that the same exact issue happens on both Android and iOS?

[liorshk]

@ryanheise Yes, same behavior happens on both Android and iOS. Since I control the server side in this project, I am open to trying other live streaming techniques like Dash streaming. Are you aware of any live streaming methodology that works? The requirements are similar to that of EXT-X-PLAYLIST-TYPE:EVENT, which is - all previous segments should be tracked, and the final duration is not known unless some end tag exists. (basically hls fits perfectly..)

Note that I also tried LockCachingAudioSource and it also didn't work, so it seems like that the localhost proxy server you mentioned doesn't solve the issue.

[ryanheise]

I only suspected the proxy as a possible issue since it rewrites the m3u8 file on the fly, so if you're not using the proxy, it would be more likely to work I would think.

The next thing to investigate is specifically the involvement of ConcatenatingAudioSource. What behaviour do you notice when not using this, and just doing setUrl? Do you still get something similar to issue 1?

[liorshk]

@ryanheise If I understand you correctly, then I tried main.dart in your example repository which doesn't use ConcatenatingAudioSource, and only plays a single track. On my repository, I managed to create my own ConcatenatingAudioSource equivalent which doesn't move to the next tracks by just maintaining one AudioSource at a time.

However, it seems like there is a different issue now that involves the fact that when the .m3u8 file is updated with more tracks/segments, the buffering is not updated so it's just stuck at ProcessingState.buffering.

In addition, regarding the 2nd issue I mentioned in the original post, it still happens even without ConcatenatingAudioSource. It seems like there is some kind of memory regarding which segment to load out of the m3u8 segments that is kept in the underlying system, because it remembers it even if I replace the audio source, is that possible?

[liorshk]

@ryanheise The buffering issue is caused because the library is currently not handling the "Stalling" event on IOS. I believe I solved it with this fix: https://github.com/ryanheise/just_audio/commit/987c586d22812c420d5c013ae10626671663576c

Other than that, it seems like ConcatenatingAudioSource does have an issue with m3u8 files like mentioned in the original post. When using my own wrapper over AudioPlayer, it seems to work fine.

[maximeburri]

I am experiencing the same issue with a m3u8 file which is generated on the fly and does not initially have the #EXT-X-ENDLIST tag. If the file is loaded after it has ended and the tag is there, it works as expected.

Do you plan to create a PR for this issue?

[ryanheise]

Apologies for not reporting back earlier on this, but I have not found anything within the code that could cause this since on both iOS and Android, the HLS playback behaviour is entirely controlled by the native API provided by the platform. It's conceivable that this is a limitation of the two platforms. However, it is also conceivable that the platforms can play HLS streams without #EXT-X-ENDLIST at the end but that they may just be sensitive to any inconsistencies in the metadata such as the durations for each segment and so on.

While I can't see anything I can do in code (since it is handled internally by the platform), it could be worth double checking all of the metadata including what is in the m3u8 file itself and the individual audio files.

[liorshk]

@maximeburri on my end, the issue was only in IOS and it occured because of the demand for updating the m3u8 file every 1.5*target duration. It seems to be part of the m3u8 rfc, but its enforced only in ios.

[ryanheise]

That's interesting, here is the relevant paragraph for anyone who runs into similar issues dynamically generating their own HLS stream:

If a Media Playlist does not contain the EXT-X-ENDLIST tag, the server MUST make a new version of the Playlist file available that contains at least one new Media Segment. It MUST be made available relative to the time that the previous version of the Playlist file was made available: no earlier than one-half the target duration after that time, and no later than 1.5 times the target duration after that time. This allows clients to utilize the network efficiently.

[maximeburri]

Thanks for pointing this out @ryanheise. Indeed, by adding the new segments earlier than 1.5 times the target duration, the player seems to work correctly.

[ryanheise]

You can thank @liorshk 100% for the tip. :+1:

I'll close this bug now that the solution has been found outside of the plugin code.

[github-actions[bot]]

This issue has been automatically locked since there has not been any recent activity after it was closed. Please open a new issue for related bugs, or use StackOverflow if you need help with just_audio.