mp4 with a height of 1080 does not return correct widthHeightRatio on a device screen that is 480x800

[42footPaul]

I've been implementing ExoPlayer in our Adobe AIR project as an ANE in order to do progressive download and save of mp4 files. This is working rather well and the player seems to work like a charm for our case. I'm using master and used the demo player as a start and added functionality as we needed such as the controls UI and a custom HttpDataSource and TransferListener to save the files.

In our latest testing we are getting a video scaling issue on devices with 480x800 screen resolution with videos that are 1080 in height.

I don't have access to the device at the moment so all I can do is have our tester run it and email our own logs.

For the 800x480 device we get this back for a video that is 1920x1080 PlayerActivity::VIDEO SURFACE CHANGED format=4 w=800 h=480 PlayerActivity::VIDEO SIZE CHANGED w=1920 h=540 widthRatio=1.0 PlayerActivity::widthHeightRatio=3.5555556

this is what we get for a video that is 1440x1080 PlayerActivity::VIDEO SURFACE CHANGED format=4 w=800 h=480 PlayerActivity::VIDEO SIZE CHANGED w=1440 h=540 widthRatio=1.0 PlayerActivity::widthHeightRatio=2.6666667

On a device that is 854x480 we get this back for the exact same 1920x1080 video PlayerActivity::VIDEO SURFACE CHANGED format=4 w=854 h=480 PlayerActivity::VIDEO SIZE CHANGED w=1920 h=1080 widthRatio=1.0 PlayerActivity::widthHeightRatio=1.7777778

We verified that this happens in the Demo App as well. Any suggestions here?

[ojw28]

What exactly do you see in the demo app? I'm not sure where you've added logging. Are you seeing an incorrect height reported to PlayerActivity.onVideoSizeChanged (540 instead of 1080)?

Is this a problem on multiple devices with that resolution, or just a single device? I suspect it's more likely to be a device specific than a problem that's caused by the size of the screen itself. Given the device only has a 480p screen, I also wouldn't be at all surprised if the hardware isn't really powerful enough to play 1080p content. What device is it?

[42footPaul]

The screenshots I have attached are from the demo app. We are in fact seeing an incorrect height reported to PlayerActivity.onVideoSizeChanged (540 in that case instead of 1080) but the widthHeightRatio=3.5555556 is also very wrong and I suspect this may be part of the issue?

We are seeing this on multiple devices Karbonn A34 Lite JellyBean 512MB 480x800 Karbonn A99i JellyBean 512MB 480x800 Gionee Pioneer P2 JellyBean 512MB 480x800

They play the videos just fine it's just that they are rendered with an incorrect height. As you can see by the screenshots they are squeezed vertically. These devices play the videos outside of the demo app with whatever media player it uses just fine.

I am part of a remote team and don't have direct access to the devices otherwise I could send better logs. If there is something specific you want me to log I can get that done.

For what it's worth the videos render correctly on tis device Lava iris 500 KitKat 512MB 480x854

[ojw28]

I don't know what widthHeightRatio=3.5555556 is in this context, because I don't know where you've added logging. Where does that come from?

[42footPaul]

my bad, that was unclear - that is actually the height calculation

(width * pixelWidthAspectRatio) / height

public void onVideoSizeChanged(int width, int height, float pixelWidthAspectRatio) { shutterView.setVisibility(View.GONE); surfaceView.setVideoWidthHeightRatio(height == 0 ? 1 : (width * pixelWidthAspectRatio) / height); Log.d(TAG, "VIDEO SIZE CHANGED"); }

[ojw28]

Right, got it. So the only thing wrong is that the height is half what it should be. If it were correct, then the widthHeightRatio would be correct too.

[42footPaul]

right - so it seems that the video height is being reported incorrectly?

[ojw28]

Please try adding logging to MediaCodecVideoTrackRenderer.onOutputFormatChanged. Specifically, if you could get the values of KEY_WIDTH, KEY_HEIGHT, and (if they exist) the values of the four KEY_CROP_X entries.

Please also add logging to MediaCodecVideoTrackRenderer.onInputFormatChanged to print holder.format.width, holder.format.height and holder.format.pixelWidthHeightRatio.

My guess would be that the decoder is downscaling the video by a factor of 2 in the vertical direction because these devices can't support 1080p buffers. The results of the suggested debugging above should help to validate this theory.

[42footPaul]

Here's the logs you were asking for - the interesting thing is that this function seems to get called twice in a row only for these 1080 vids. (log shows up twice back to back with exact same results - didn't bother pasting the 2nd output)

MediaCodecVideoTrackRenderer::onOutputFormatChanged MediaCodecVideoTrackRenderer::KEY_WIDTH = 1440 MediaCodecVideoTrackRenderer::KEY_HEIGHT = 540 MediaCodecVideoTrackRenderer::KEY_CROP_RIGHT = 1439 MediaCodecVideoTrackRenderer::KEY_CROP_LEFT = 0 MediaCodecVideoTrackRenderer::KEY_CROP_TOP = 0 MediaCodecVideoTrackRenderer::KEY_CROP_BOTTOM = 539

[42footPaul]

Any suggestions on this?

[42footPaul]

Realized I didn't give you the input format changed log you were looking for.

MediaCodecVideoTrackRenderer::onInputFormatChanged MediaCodecVideoTrackRenderer::holder.format.width = 1920 MediaCodecVideoTrackRenderer::holder.format.height = 1080 MediaCodecVideoTrackRenderer::holder.format.pixelWidthHeightRatio = 1.0

[42footPaul]

doing some more digging.

this is a list of the video decoders on the device OMX.MTK.VIDEO.DECODER.MPEG4 OMX.MTK.VIDEO.DECODER.H263 OMX.MTK.VIDEO.DECODER.AVC OMX.MTK.VIDEO.DECODER.VPX OMX.MTK.VIDEO.DECODER.VC1 OMX.MTK.VIDEO.DECODER.DIVX OMX.MTK.VIDEO.DECODER.DIVX3 OMX.MTK.VIDEO.DECODER.XVID

The decoder chosen is OMX.MTK.VIDEO.DECODER.AVC

Since the stock Video Player on the device plays the video correctly could it be using the MPEG4 decoder?

[42footPaul]

This is a bit of a blocker in our next release and need to find out if you think this is something that can be addressed in the framework.

[42footPaul]

More investigation OMX.MTK.VIDEO.DECODER.AVC supports the following profiles profile:1 profile:2 profile:4 profile:8 profile:16 profile:32 profile:64 profile:128 profile:256 profile:512 profile:1024 profile:2048 profile:4096 profile:8192 profile:16384 profile:32768

when I convert the decimal values here to hex and compare them with the table of supported types in OpenMax found here https://www.khronos.org/files/openmax_il_spec_1_0.pdf on page 285/286 I see that this decoder supports up to and including Level 5.1 which when compared to the list found here https://en.wikipedia.org/wiki/H.264/MPEG-4_AVC#Levels tells me the decoder itself supports 1080.

You are saying that these devices can't support 1080p buffers yet the 1080 vids play fine on the stock player and it looks to me like the decoder supports 1080. I don't understand. The issue is still open and flagged with need more info. Is there anything I can do to further the cause?

[ojw28]

It looks to me like the decoder just chooses to decode it into a half height texture, most likely due to memory limitations of the device.

• If that's the only problem, you could just change MediaCodecVideoTrackRenderer to take the width and height from the input format instead (i.e. have it set currentWidth, currentHeight and currentPixelWidthHeightRatio in onInputFormatChanged rather than in onOutputFormatChanged). That should result in the correct aspect ratio being reported, and hence the surface being sized correctly. You could also implement any other workaround at all that causes sensible values to be sent to onVideoSizeChanged (e.g. if you know the dimensions from somewhere else).
• As an aside, decoding 1080p content on a 480p device needlessly drains battery and (if streamed over the network) data. Your users will get a much better experience if you give these devices something nearer to 480p video, which likely also solves this problem because I suspect the decoder will not render smaller videos in this way.