search

google / ExoPlayer

An extensible media player for Android
Apache License 2.0
21.65k stars 6.01k forks source link

Dolby Atmos Stream not playing for Device supporting Dolby Atmos #10937

Open khushank opened 1 year ago

khushank commented 1 year ago

ExoPlayer Version

2.18.0

Devices that reproduce the issue

Xiaomi MiTV-MOOQ0 Android 10

Model Name: 43 Y1S Pro Android 10 Model C00

Devices that do not reproduce the issue

Fire TV Stick 4K - 1st Gen (2018) Android 7.1.2

Reproducible in the demo app?

Yes

Reproduction steps

Expected result

Actual result

Observe that Video is stuck at 1st frame only.

Media

We will share the Media details via email to [dev.exoplayer@gmail.com]

Bug Report

khushank commented 1 year ago

Attaching the adb logs and video of the issue for reference.

https://user-images.githubusercontent.com/1241576/213659699-57681d84-64d6-4f78-8a80-94d7268a1c8d.mp4

4k_dolby_exo.txt

Bug Report bugreport-croods-QTG3.200305.006-2023-01-20-14-41-59.zip

khushank commented 1 year ago

Please note that we are using the below Android API to detect whether Dolby Atmos Decoder(EAC3-JOC) is supported by the device or not.


Log.i("AudioTrack", "Getting EAC3 support in Android 10 and above ")
    Log.i(
        "AudioTrack", "AudioTrack support in Android 10 and above = " +
                AudioTrack.isDirectPlaybackSupported(
                    AudioFormat.Builder().setEncoding(AudioFormat.ENCODING_E_AC3_JOC)
                        .build(),
                    AudioAttributes.Builder().setUsage(AudioAttributes.USAGE_MEDIA).build()
                )
    )

The API returns false, but the manufacturer claims that it supports Dolby Atmos. As we are getting encoder support in Event Logs

group [ [X] Track:0, id=audio/hin/ec-3, mimeType=audio/eac3-joc, bitrate=449314, codecs=ec+3, drm=[playready,widevine,cenc], channels=6, sample_rate=48000, language=hi, label=Hindi, supported=YES ]

microkatz commented 1 year ago

@khushank thank you for filing your issue and sending the media content through. I tried reproducing with the content you sent but received a HTTP Error 403 with the license URI that you provided. Is there other content you can point us to in order to reproduce the issue?

khushank commented 1 year ago

@microkatz I have shared the updated Video url over the email which have a long expiry durations. Please check the same and let us know if any issues.

khushank commented 1 year ago

@microkatz can you please confirm , if you have got the updated url and now able to proceed further on this.

microkatz commented 1 year ago

Yes. I received the updated url and am able to proceed further on this. Thank you!

microkatz commented 1 year ago

@khushank Looking through the logs in your attached bug report from the email, the decoder OMX.google.raw.decoder was selected but I don't believe it is capable of decoding the eac3-joc.

EventLogger: audioDecoderInitialized [eventTime=0.89, mediaPos=1178.71, window=0, period=0, OMX.google.raw.decoder]

Unfortunately, we have not been able to reproduce the issue with our devices and your supplied media sample. Our devices with your supplied sample have initialized with OMX.amlogic.audio.decoder.eac3.

Would you be able to debug the scenario yourself by utilize the demo app on this project? You can insert your media into the list in media.exolist.json. We would need a clearer picture into MediaCodecAudioRenderer::supportsFormat. Namely the list of decoderInfos retrieved from getDecoderInfos for the the supportsFormat call on the Atmos track. In addition, the decoders retrieved from MediaCodecRenderer::getAvailableCodecInfos.

khushank commented 1 year ago

@microkatz Thanks for this information. Can you please clarify on what device model you were able to play the content , we will also confirm the same on those devices.

Wanted to confirm that The attached logs were taken in the latest Sample Exoplayer app by inserting the media in media.exolist.json file

Also from logs we can see the Exoplayer Event logs. Can you please confirm Does the below line Specifies that eac3-joc decoder is supported by the device or not?

[X] Track:0, id=audio/hin/ec-3, mimeType=audio/eac3-joc, bitrate=449314, codecs=ec+3, drm=[playready,widevine,cenc], channels=6, sample_rate=48000, language=hi, label=Hindi, supported=YES

Can you please point out how to exactly use the API's MediaCodecAudioRenderer::supportsFormat , getDecoderInfos, getAvailableCodecInfos to get the list of codecs in the device.

01-20 14:07:38.425 31436 31436 D EventLogger:   group [
01-20 14:07:38.425 31436 31436 D EventLogger:     [X] Track:0, id=video/hev1/6, mimeType=video/hevc, bitrate=1663005, codecs=hev1.2.4.H150.b0, drm=[playready,widevine,cenc], res=1280x720, fps=25.0, supported=YES
01-20 14:07:38.426 31436 31436 D EventLogger:     [X] Track:1, id=video/hev1/5, mimeType=video/hevc, bitrate=2480104, codecs=hev1.2.4.H150.b0, drm=[playready,widevine,cenc], res=1920x1080, fps=25.0, supported=YES
01-20 14:07:38.426 31436 31436 D EventLogger:     [X] Track:2, id=video/hev1/4, mimeType=video/hevc, bitrate=3086522, codecs=hev1.2.4.H150.b0, drm=[playready,widevine,cenc], res=1920x1080, fps=25.0, supported=YES
01-20 14:07:38.426 31436 31436 D EventLogger:     [X] Track:3, id=video/hev1/3, mimeType=video/hevc, bitrate=4675006, codecs=hev1.2.4.H150.b0, drm=[playready,widevine,cenc], res=2560x1440, fps=25.0, supported=YES
01-20 14:07:38.426 31436 31436 D EventLogger:     [ ] Track:4, id=video/hev1/2, mimeType=video/hevc, bitrate=7870667, codecs=hev1.2.4.H150.b0, drm=[playready,widevine,cenc], res=3840x2160, fps=25.0, supported=YES
01-20 14:07:38.426 31436 31436 D EventLogger:     [ ] Track:5, id=video/hev1/1, mimeType=video/hevc, bitrate=9883502, codecs=hev1.2.4.H150.b0, drm=[playready,widevine,cenc], res=3840x2160, fps=25.0, supported=YES
01-20 14:07:38.426 31436 31436 D EventLogger:   ]
01-20 14:07:38.426 31436 31436 D EventLogger:   group [
01-20 14:07:38.426 31436 31436 D EventLogger:     [X] Track:0, id=audio/hin/ec-3, mimeType=audio/eac3-joc, bitrate=449314, codecs=ec+3, drm=[playready,widevine,cenc], channels=6, sample_rate=48000, language=hi, label=Hindi, supported=YES
01-20 14:07:38.426 31436 31436 D EventLogger:   ]
khushank commented 1 year ago

@microkatz I got this list of Audio decoders supported by device using Android API : https://developer.android.com/reference/android/media/MediaCodecList

This list shows availability of OMX.MS.AC3.Decoder in the device but for a Dolby Atmos content encoded with (eac3-joc) Exoplayer is picking OMX.google.raw.decoder

Supported Types = [audio/ac3, audio/eac3] Codec Name = OMX.MS.AC3.Decoder Codec Canonical Name = OMX.MS.AC3.Decoder Codec Name = OMX.MS.AC3.Decoder Codec Canonical Name = OMX.MS.AC3.Decoder

`
Supported Types = [audio/mp4a-latm]
Codec Name = OMX.google.aac.decoder
Codec Canonical Name = OMX.google.aac.decoder
Supported Types = [audio/3gpp]
Codec Name = OMX.google.amrnb.decoder
Codec Canonical Name = OMX.google.amrnb.decoder
Supported Types = [audio/amr-wb]
Codec Name = OMX.google.amrwb.decoder
Codec Canonical Name = OMX.google.amrwb.decoder
Supported Types = [audio/flac]
Codec Name = OMX.google.flac.decoder
Codec Canonical Name = OMX.google.flac.decoder
Supported Types = [audio/g711-alaw]
Codec Name = OMX.google.g711.alaw.decoder
Codec Canonical Name = OMX.google.g711.alaw.dec
Supported Types = [audio/g711-mlaw]
Codec Name = OMX.google.g711.mlaw.decoder
Codec Canonical Name = OMX.google.g711.mlaw.dec
Supported Types = [audio/mpeg]
Codec Name = OMX.google.mp3.decoder
Codec Canonical Name = OMX.google.mp3.decoder
Supported Types = [audio/opus]
Codec Name = OMX.google.opus.decoder
Codec Canonical Name = OMX.google.opus.decoder
Supported Types = [audio/raw]
Codec Name = OMX.google.raw.decoder
Codec Canonical Name = OMX.google.raw.decoder
Supported Types = [audio/vorbis]
Codec Name = OMX.google.vorbis.decoder
Codec Canonical Name = OMX.google.vorbis.decode
Supported Types = [audio/mp4a-latm]
Supported Types = [audio/3gpp]
Supported Types = [audio/amr-wb]
Supported Types = [audio/flac]
Supported Types = [audio/ac3, audio/eac3]
Codec Name = OMX.MS.AC3.Decoder
Codec Canonical Name = OMX.MS.AC3.Decoder
Codec Name = OMX.MS.AC3.Decoder
Codec Canonical Name = OMX.MS.AC3.Decoder
Supported Types = [audio/x-adpcm-ima, audio/x-a
Codec Name = OMX.MS.ADPCM.Decoder
Codec Canonical Name = OMX.MS.ADPCM.Decoder
Codec Name = OMX.MS.ADPCM.Decoder
Codec Canonical Name = OMX.MS.ADPCM.Decoder
Supported Types = [audio/flac]
Codec Name = OMX.MS.FLAC.Decoder
Codec Canonical Name = OMX.MS.FLAC.Decoder
Supported Types = [audio/mp4a-latm-hw]
Codec Name = OMX.MS.AAC.Decoder
Codec Canonical Name = OMX.MS.AAC.Decoder
Supported Types = [audio/mpeg-L1]
Codec Name = OMX.MS.MP1.Decoder
Codec Canonical Name = OMX.MS.MP1.Decoder
Supported Types = [audio/mpeg-L2]
Codec Name = OMX.MS.MP2.Decoder
Codec Canonical Name = OMX.MS.MP2.Decoder
Supported Types = [audio/mpeg]
Codec Name = OMX.MS.MP3.Decoder
Codec Canonical Name = OMX.MS.MP3.Decoder
Supported Types = [audio/vnd.rn-realaudio]
Codec Name = OMX.MS.RA.Decoder
Codec Canonical Name = OMX.MS.RA.Decoder
Supported Types = [audio/vorbis]
Codec Name = OMX.MS.VORBIS.Decoder
Codec Canonical Name = OMX.MS.VORBIS.Decoder
Supported Types = [audio/x-ms-wma, audio/x-ms-w
Codec Name = OMX.MS.WMA.Decoder
Codec Canonical Name = OMX.MS.WMA.Decoder
Codec Name = OMX.MS.WMA.Decoder
Codec Canonical Name = OMX.MS.WMA.Decoder`
microkatz commented 1 year ago

@khushank. I tested with the Xiaomi Mi Box 4K Model: MDz-22-AB among some odd Pixels and Samsung devices.

Can you please point out how to exactly use the API's MediaCodecAudioRenderer::supportsFormat , getDecoderInfos, getAvailableCodecInfos to get the list of codecs in the device.

Please clone Github Repo. Insert the media url into the media.exolist.json. Open the file MediaCodecAudioRenderer.java and go to the method supportsFormat. Put a breakpoint at the beginning of the method. Run the demo app in debugger mode and play your media file. It should stop at your breakpoint. It would be great to see the data returned from getDecoderInfos at line ~308 and the boolean value assigned to isFormatSupported at ~319. You can either put breakpoints on those lines or step through the method. These lines are in the snippet cut from the file below.

List<MediaCodecInfo> decoderInfos =
        getDecoderInfos(mediaCodecSelector, format, /* requiresSecureDecoder= */ false, audioSink);
  if (decoderInfos.isEmpty()) {
    return RendererCapabilities.create(C.FORMAT_UNSUPPORTED_SUBTYPE);
  }
  if (!supportsFormatDrm) {
    return RendererCapabilities.create(C.FORMAT_UNSUPPORTED_DRM);
  }
  // Check whether the first decoder supports the format. This is the preferred decoder for the
  // format's MIME type, according to the MediaCodecSelector.
  MediaCodecInfo decoderInfo = decoderInfos.get(0);
  boolean isFormatSupported = decoderInfo.isFormatSupported(format);
  boolean isPreferredDecoder = true;
khushank commented 1 year ago

@microkatz I checked this and while playing the Sample dolby Atmos content, it returns from following

 if (!MimeTypes.isAudio(format.sampleMimeType)) {
      return RendererCapabilities.create(C.FORMAT_UNSUPPORTED_TYPE);
    }

Can you please confirm on my query in previous comments

Also from logs we can see the Exoplayer Event logs. Can you please confirm Does the below line Specifies that eac3-joc decoder is supported by the device or not?

[X] Track:0, id=audio/hin/ec-3, mimeType=audio/eac3-joc, bitrate=449314, codecs=ec+3, drm=[playready,widevine,cenc], channels=6, sample_rate=48000, language=hi, label=Hindi, supported=YES

01-20 14:07:38.425 31436 31436 D EventLogger:   group [
01-20 14:07:38.425 31436 31436 D EventLogger:     [X] Track:0, id=video/hev1/6, mimeType=video/hevc, bitrate=1663005, codecs=hev1.2.4.H150.b0, drm=[playready,widevine,cenc], res=1280x720, fps=25.0, supported=YES
01-20 14:07:38.426 31436 31436 D EventLogger:     [X] Track:1, id=video/hev1/5, mimeType=video/hevc, bitrate=2480104, codecs=hev1.2.4.H150.b0, drm=[playready,widevine,cenc], res=1920x1080, fps=25.0, supported=YES
01-20 14:07:38.426 31436 31436 D EventLogger:     [X] Track:2, id=video/hev1/4, mimeType=video/hevc, bitrate=3086522, codecs=hev1.2.4.H150.b0, drm=[playready,widevine,cenc], res=1920x1080, fps=25.0, supported=YES
01-20 14:07:38.426 31436 31436 D EventLogger:     [X] Track:3, id=video/hev1/3, mimeType=video/hevc, bitrate=4675006, codecs=hev1.2.4.H150.b0, drm=[playready,widevine,cenc], res=2560x1440, fps=25.0, supported=YES
01-20 14:07:38.426 31436 31436 D EventLogger:     [ ] Track:4, id=video/hev1/2, mimeType=video/hevc, bitrate=7870667, codecs=hev1.2.4.H150.b0, drm=[playready,widevine,cenc], res=3840x2160, fps=25.0, supported=YES
01-20 14:07:38.426 31436 31436 D EventLogger:     [ ] Track:5, id=video/hev1/1, mimeType=video/hevc, bitrate=9883502, codecs=hev1.2.4.H150.b0, drm=[playready,widevine,cenc], res=3840x2160, fps=25.0, supported=YES
01-20 14:07:38.426 31436 31436 D EventLogger:   ]
01-20 14:07:38.426 31436 31436 D EventLogger:   group [
01-20 14:07:38.426 31436 31436 D EventLogger:     [X] Track:0, id=audio/hin/ec-3, mimeType=audio/eac3-joc, bitrate=449314, codecs=ec+3, drm=[playready,widevine,cenc], channels=6, sample_rate=48000, language=hi, label=Hindi, supported=YES
01-20 14:07:38.426 31436 31436 D EventLogger:   ]

MicrosoftTeams-image (46)

MicrosoftTeams-image (45)

microkatz commented 1 year ago

@khushank

Can you please confirm on my query in previous comments

Also from logs we can see the Exoplayer Event logs. Can you please confirm Does the below line Specifies that eac3-joc decoder is supported by the device or not? [X] Track:0, id=audio/hin/ec-3, mimeType=audio/eac3-joc, bitrate=449314, codecs=ec+3, drm=[playready,widevine,cenc], channels=6, sample_rate=48000, language=hi, label=Hindi, supported=YES

In essence it means that the decoder provided by getDecoderInfos in that supportsFormat said it supported that format.

Thank you for going through the debug process and providing the images and data. It looks like the code is checking a video track rather than the audio? If you notice, the Format is of type video/mp4, has a height value, etc. That is why it returns in that conditional as its not an Audio type. You need to skip forward till you are in that method and the format attributes match that audio/eac3-joc track.

khushank commented 1 year ago

Thanks @microkatz For the details. I again did the debugging to get the information related to Audio decoders. getDecoderInfos Method provides only 1 audio decoder : "OMX.google.raw.decoder"

Looking at all this data can you please clarify why Exoplayer is not picking the right hardware decoder: OMX.MS.AC3.Decoder

decoder

audioSink.supportsFormat(format) returns true for the One Plus device and also the AudioSync Object returns AudioFormat.ENCODING_E_AC3_JOC ,in supportedCodecs Array which signifies that EAC3-JOC decoder is supported by the device.

audioSink

Capture

Format of the codec captured while debugging.

format

khushank commented 1 year ago

@microkatz Any further update on this. I hope i have now provided relevant information for debugging this further.

microkatz commented 1 year ago

@khushank

Thank you for your patience. In digging it seems that the OMX.google.raw.decoder is selected because the device says that it supports the eac3-joc format directly but it needs a decoder do decrypt the content. The logs I was looking at were from your DRM scenario where the decoder is only initialized to decrypt the media and then the eac3-joc is sent directly to the AudioTrack.

Just to have a bit more clarity, could you send trace logs showcasing the issue with the unencrypted media url you sent? You can send via the same email thread to dev.exoplayer@gmail.com.

khushank commented 1 year ago

@microkatz Thanks for the reply . PFA Logs while playing the non drm content on One Plus TV. Curious to see that there are no AudioDecoder Initialization logs and Event logger doesn't show even OMX.google.raw.decoder getting selected.

Exologs.txt

bugreport-OnePlusTV-QTG3.201207.001-2023-02-16-18-00-55.zip

khushank commented 1 year ago

Hi @microkatz Any updates on this. Client is pushing on this as it wants to make live the dolby atmos for contents and this issue is holding a major release for them.

microkatz commented 1 year ago

@khushank.

Can you identify this One Plus device? It is also experiencing the same issue as the Xiaomi device you originally filed the issue with?

Unfortunately we have not been able to reproduce the issue ourselves but it seems to be a device issue. The device says it supports direct playback of the audio but then cannot play it. That is why there is not an audiodecoder initialization as the device says it supports the format directly.

Earlier you posted here that you checked AudioTrack.isDirectPlaybackSupported? Could you run that method again for your eac3-joc as well as just eac3?

khushank commented 1 year ago

@microkatz The issue is also reproducible with OnePlus TV device : Model Name: 43 Y1S Pro Android 10 Model C00 ![Uploading OnePlusTV.png…]()

microkatz commented 1 year ago

@khushank You posted https://github.com/google/ExoPlayer/issues/10937#issuecomment-1398132317 that you checked AudioTrack.isDirectPlaybackSupported? Could you run that method again with eac3 rather than eac3-joc? See if it returns false?

khushank commented 1 year ago

@microkatz Just Updating for OnePlus TV device There are 2 methods by which I'm trying to detect the EAC3 and EAC3-JOC support

Method 1 for Android 10 & above

AudioTrack.isDirectPlaybackSupported(
                            AudioFormat.Builder().setEncoding(AudioFormat.ENCODING_E_AC3)
                                .build(),
                            AudioAttributes.Builder().setUsage(AudioAttributes.USAGE_MEDIA).build()
                        )

Method 2 for Android 9 and Below

val audioDevices: Array<AudioDeviceInfo> =
            manager.getDevices(AudioManager.GET_DEVICES_OUTPUTS)
        for (audioDevice in audioDevices) {
            Log.i("AudioTrack", "Audio Device Type = ${audioDevice.type}")
            Log.i("AudioTrack", "Audio Format Supported = ${audioDevice.encodings.contentToString()}")
            if (audioDevice.type == AudioDeviceInfo.TYPE_BUILTIN_SPEAKER &&
                audioDevice.encodings.contains(AudioFormat.ENCODING_E_AC3)
            ) {
                Log.i("AudioTrack EAC3 Support", "EAC3 supported!! ")
            }

        }

Following is the behavior for different codecs

Codec | isDirectPlaybackSupported (Method 1) | audioDevice.encodings. (Method 2) -- | -- | -- ENCODING_E_AC3-JOC | FALSE | TRUE ENCODING_E_AC3 | FALSE | TRUE

This means that with new API it says that it doesn't support and with Old API it supports which is contradictory.

microkatz commented 1 year ago

Hmmm. Looking back at https://github.com/google/ExoPlayer/issues/10937#issuecomment-1418937113, the code path passes audioSink.supportsFormat directly. This means that AudioTrack.isDirectPlaybackSupported method must return true with some channel config. Unlike in AudioCapabilites.getMaxSupportedChannelCountForPassthrough, you don't call setChannelMask in the build of your AudioFormat which is probably why you are getting false.

It is odd that it is happening with two devices but it does seem that its a device specific issue. The device is not playing the audio track which is why nothing is progressing. Unfortunately, it can be a number of reasons. It would be easier if I could debug it myself to see why the device is not playing the samples that it is given.

A workaround strategy would be to not allow passthrough for that audio type and require it to be decoded? You can use a ForwardingAudioSink and override the supportsFormat method. If the format is eac3-joc then return false. Hopefully that at least unblocks you.

microkatz commented 1 year ago

@khushank

I'm going to close this issue as it seems that its been addressed? Please open if you have continuing questions.

khushank commented 1 year ago

@microkatz Can you please help in providing info with Which API we would come to know that Device doesn't support dolby Atmos Playback, even when device claims that is supports Atmos playback. This will be helpful if we want to black list some of the devices in the market to which we are serving the Atmos stream but still device doesn't supports it.

As currently we don't receive any Player Error just the Video freezes on the 1st frame itself.

Not sure how to reopen the issue here.

lopetube commented 1 year ago

Hello , i was following this bug tracker from weeks. I dont think its an isolated one because I've also problem with eac3 decoding . The bug I'm facing is i've no audio on Nvidia Shield Pro but it works on One Plus 8t

khushank commented 1 year ago

@microkatz can you please reopen this and try to reproduce with the device that @lopetube has provided. This might be worth looking into as we migh face problems in devices which user have and want to play dolby atmos audio.

khushank commented 1 year ago

@microkatz Also regarding your comment https://github.com/google/ExoPlayer/issues/10937#issuecomment-1442115626

Can you explain this why code snippet as I couldn't understand the same.

microkatz commented 1 year ago

@lopetube,

Hello , i was following this bug tracker from weeks. I dont think its an isolated one because I've also problem with eac3 decoding . The bug I'm facing is i've no audio on Nvidia Shield Pro but it works on One Plus 8t

You are hitting the same issue with an Nvidia Shield? A single video frame is displayed and stream seems frozen? Do you have sample media with which we can try to reproduce the issue?

lopetube commented 1 year ago

@lopetube,

Hello , i was following this bug tracker from weeks. I dont think its an isolated one because I've also problem with eac3 decoding . The bug I'm facing is i've no audio on Nvidia Shield Pro but it works on One Plus 8t

You are hitting the same issue with an Nvidia Shield? A single video frame is displayed and stream seems frozen? Do you have sample media with which we can try to reproduce the issue?

Hello , not same issue video has no problem and is being displayed properly but instead I've no sound (on NVIDIA shield) . I realized this beahavior was for EAC3 audio streams (checked with ffmpeg) . Otherwise the same stream work properly (video and audio) on Oneplus 8T

microkatz commented 1 year ago

@lopetube

Since it is a different problem you are facing then if you could please create a new github issue? The issue template can be found here. If you're unable to share bug reports or test content publicly, please send them to dev.exoplayer@gmail.com using a subject in the format "Issue #1234" ("#1234" is replaced with your new issue number).

jmvillagra commented 11 months ago

I have the same issue with a Dolby digital Plus (codecs="ec-3"), only the first frame is decoded and the stream is frozen. I'm using exoplayer 2.18.3.

It plays on some screens/tvs and stays frozen is some others.