search

google / ExoPlayer

This project is deprecated and stale. The latest ExoPlayer code is available in https://github.com/androidx/media
https://developer.android.com/media/media3/exoplayer
Apache License 2.0
21.7k stars 6.01k forks source link

Support gapless playback of Ogg Vorbis #3470

Open th3hamm0r opened 6 years ago

th3hamm0r commented 6 years ago

Issue description

We are facing problems with click noises when playing a sequence of ogg/vorbis files with the ExoPlayer. The interesting thing is, that I was able to create a quick&dirty player with Android's decoding-APIs where the playback is absolutely gapless. So what is the difference to the ExoPlayer and is there a way to avoid those clicking sounds? Eg. do I have to pass some configuration or override the DynamicConcatenatingMediaSource in a way to avoid this?

Note: There are several issues, including https://github.com/google/ExoPlayer/issues/497 where I actually stated that the playback is gapless. Most of the time this is true, but in some cases the transition between files result in a click noise.

Reproduction steps

  1. Clone my test-app from https://github.com/th3hamm0r/exoplayer-gapless-test
  2. Compare the sound of the ExoPlayer vs. the custom player. Both implementations play two 10s files created from gapless wav-files.
  3. In my tests the ExoPlayer produced a click noise when transitioning (after 10s), the CustomPlayer did not.

Note: These two files also play gapless with several audio tools (Audacity, Aqualung player,...).

Link to test content

Note: The custom player is a quick&dirty solution created from several sources with some deprecated APIs and so on...

https://github.com/th3hamm0r/exoplayer-gapless-test

Version of ExoPlayer being used

2.5.4

Device(s) and version(s) of Android being used

Tested on multiple devices, eg. Nexus 5 (6.0.1), Nexus 5X (8.0), Nexus 9 (7.1.1), Nokia 6,...

A full bug report captured from the device

--

th3hamm0r commented 6 years ago

My first thought was, that maybe there is a difference in how the codec is re-initialized, so I already tried to dig into the MediaCodecRenderer/MediaCodecAudioRenderer a bit to find some major differences. I failed, so I hope you can help here.

andrewlewis commented 6 years ago

For reference, the streams are 1.ogg and 2.ogg.

Ogg containers have gapless playback information that I think we don't currently handle. There is some information about how this works here. To summarize, each Ogg page containing a completed Vorbis audio packet has a granule position that specifies the end PCM sample index of the last completed audio packet in the page. At the start of the stream, we may need to discard PCM audio samples that have an inferred position before 0. For trimming the end of the stream, the granule position in the last page may indicate a last PCM sample index that is before the last decoded PCM sample.

I'll mark this as an enhancement to provide support but it's unlikely we will get round to it soon: for other formats where we support gapless playback, we know the encoder delay/padding when we parse the header of the file, but it looks like we can't do this for Ogg so it may be difficult to implement. We do already support gapless playback of MP3 (based on Xing/ID3 headers) and MP4 (udta and edit lists) containers, so perhaps you can switch to use one of those. Also, I think opus bitstreams contain gapless playback information in the codec initialization data, so those should just work with both MediaCodecAudioRenderer and the opus extension.

th3hamm0r commented 6 years ago

@andrewlewis Thanks for your quick response! Would this be an enhancement of ExoPlayer's OggExtractor? If this is the case, do you think it would be possible to somehow use Android's MediaExtractor feeding the ExoPlayer to fix this problem, till you fix this in the ExoPlayer? Or can you think of another (quick-)fix to get a gapless playback of ogg/vorbis with the player?

Even with lame-mp3 encoded wav-segments we still have click sounds for some reason. Opus is nice, but encoding existing wav-segments is not as easy as with ogg/vorbis. Without a special handling (http://lists.xiph.org/pipermail/opus/2017-November/004031.html), those segments don't play gapless (tested with the ExoPlayer).

andrewlewis commented 6 years ago

It's an enhancement to OggExtractor, though might need changes elsewhere as well, unless we can find a way to get the encoder delay/padding before we first create the Format.

We used to provide a MediaExtractor wrapper in v1 (FrameworkSampleSource) but there were fundamental issues with it.

I can't think of any quick fixes that do this properly. On the off chance you want gapless playback of some particular fixed files, you could do a hacky fix where you find out how many samples need to be trimmed (perhaps based on kKeyValidSamples in the MediaFormat? I'm not sure) then make a custom Extractor wrapping OggExtractor that provides the known values. But in that case you may as well change to another container/codec.

Gapless with MP3 should work fine. If you think it's a bug in ExoPlayer please file a separate issue and I'll take a look. Thanks.

Aside: I had a quick look at how the framework does this. It seems to append a number of valid samples to input buffers sent to the decoder.

th3hamm0r commented 6 years ago

@andrewlewis Again, thanks a lot for your quick reply and your help! I've created another issue for MP3 (#3475), it looks like it suffers from the same problem. Unfortunately mp3 is not an option for us, so I will try to find a (hacky) solution in the meantime, but I hope that the vorbis issue will be fixed in the future!