search

nova-video-player / aos-AVP

NOVA opeN sOurce Video plAyer: main repository to build them all
Apache License 2.0
3.53k stars 201 forks source link

aspect ratio issues investigations #258

Open courville opened 4 years ago

courville commented 4 years ago

On Philips TV (mtk) it has been reported that AR is wrong https://www.reddit.com/r/NovaVideoPlayer/comments/dz4wh5/reqissue_android_tv_9_pie_aspect_ratio_totally/ On amlogic when selecting full screen AR is wrong with 21:9 and 4:3 videos that are stretched instead of applying a zoom. It appears when surface has been scaled and exceeds screen dimension i.e. codec does a scale instead of crop. @phhusson hinted towards https://github.com/xbmc/xbmc/blob/master/tools/android/packaging/xbmc/src/XBMCVideoView.java.in#L117 @phhusson also suggested that a echo 6 > /sys/class/video/screen_mode could help on amlogic cf. https://gist.github.com/phhusson/b36755226e06f313caca41ec573b8e93 i.e.:

#define  SCREEN_MODE_NORMAL           0
#define  SCREEN_MODE_FULL_STRETCH     1
#define  SCREEN_MODE_4_3              2
#define  SCREEN_MODE_16_9             3
#define  SCREEN_MODE_NONLINEAR        4
#define  SCREEN_MODE_NORMAL_NOSCALEUP 5
#define  SCREEN_MODE_CROP_FULL        6
#define  SCREEN_MODE_CROP             7
courville commented 4 years ago

In java world one could use the MediaCodec https://developer.android.com/reference/android/media/MediaCodec#setVideoScalingMode(int)

courville commented 4 years ago

In Ndk domain, AMEDIAFORMAT_KEY_DISPLAY_CROP = "crop" exists but has been introduced in 28 (frameworks/av/media/ndk/NdkMediaFormat.cpp and android-ndk/sysroot/usr/include/media/NdkMediaFormat.h).

courville commented 4 years ago

Relevant part in avos is native/avos/external/android/libsfdec/sfdec_ndkmediacodec.cpp:sfdec_init

phhusson commented 4 years ago

Java's setVideoScalingMode calls:

        native_window_set_scaling_mode(mSurfaceTextureClient.get(), mode);
        msg->setInt32("android._video-scaling", mode);
        (void)mCodec->setParameters(msg);

This means that the information is transmitted to both hwcomposer and codec. We can't safely access native_window_set_scaling_mode but we can send a parameter to the codec.

So we can try to use AMediaCodec_setParameters to set "android._video-scaling"'s value to the mode we want

courville commented 4 years ago

Regarding more strange AR issues as reported here https://github.com/nova-video-player/aos-AVP/issues/257 one need to check that codec report is correct (should be enforced by CTS) and this can be done monitoring INFO_FORMAT_CHANGED from avos:

diff --git a/external/android/libsfdec/sfdec_ndkmediacodec.cpp b/external/android/libsfdec/sfdec_ndkmediacodec.cpp
index 57f6c7e..fe6d488 100644
--- a/external/android/libsfdec/sfdec_ndkmediacodec.cpp
+++ b/external/android/libsfdec/sfdec_ndkmediacodec.cpp
@@ -274,7 +274,7 @@ static int sfdec_read(sfdec_priv_t *sfdec, int64_t seek, sfdec_read_out_t *read_
             read_out->size.width = sfdec->width;
             read_out->size.height = sfdec->height;
             read_out->size.interlaced = 0;
-            DBG LOG("INFO_FORMAT_CHANGED: %dx%d", sfdec->width, sfdec->height);
+            LOG("INFO_FORMAT_CHANGED: %dx%d", sfdec->width, sfdec->height);
             return 0;
         } else if (index == AMEDIACODEC_INFO_OUTPUT_BUFFERS_CHANGED) {
             DBG LOG("INFO_OUTPUT_BUFFERS_CHANGED");
courville commented 4 years ago

On another track, Philips TVs (MTK based) are showing also AR issues after upgrading to Android 9 but this is not happening on Sony TVs (MTK based too). Check Kodi references https://forum.kodi.tv/showthread.php?tid=341491 and notice that somepeople are recommending to update their Philips TV firmware since Philips is supposed to have fixed the issue in an update. In the thread they prompt the user to check https://toengel.net/philipsblog/firmware-download/

courville commented 4 years ago

/sys/class/video/screen_mode is not accessible on miproj or mibox.

courville commented 4 years ago

Cf. https://gist.github.com/courville/f64cb4cd6e6f644c52042de2ca5f1b6a Asking the codec to do the scale to fit with crop seems to make amlogic mibox s working. Need to check on miproj. Issue with the above approach is that it requires API26 because AMediaCodec_setParameters is not available on former versions.

phhusson commented 4 years ago

We need someone to test on Philips/MTK and the weird rk3328 aspect-ratio issue too

Le jeu. 12 déc. 2019 à 11:13, Marc de Courville notifications@github.com a écrit :

Cf. https://gist.github.com/courville/f64cb4cd6e6f644c52042de2ca5f1b6a Asking the codec to do the scale to fit with crop seems to make amlogic mibox s working. Need to check on miproj. Issue with the above approach is that it requires API26 because AMediaCodec_setParameters is not available on former versions.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/nova-video-player/aos-AVP/issues/258?email_source=notifications&email_token=AAAA4OVYPEAZY7AKZQEQUF3QYIFC7A5CNFSM4JY5ONH2YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOEGWE7WI#issuecomment-564940761, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAAA4OQCMDMGS3PTRBYMEULQYIFC7ANCNFSM4JY5ONHQ .

courville commented 4 years ago

We need someone to test on Philips/MTK and the weird rk3328 aspect-ratio issue too Le jeu. 12 déc.

Already done: https://www.reddit.com/r/NovaVideoPlayer/comments/dz4wh5/reqissue_android_tv_9_pie_aspect_ratio_totally/ test results soon for Philips. Regarding RK apk shared as well here: https://github.com/nova-video-player/aos-AVP/issues/257

courville commented 4 years ago

Feedback from testers: no effect on miproj, philips tv, rk box. New approach to be tested: AR would work if we trust container resolution and not the codec format output change. This could result in new bugs if container information is wrong. But this could fix firestick 4k ko, philips tv with latest firmware KO and RK KO.

courville commented 4 years ago

New approach: https://gist.github.com/courville/2fe4056f2846319f69368f80fc63c2be and test apk shared.

courville commented 4 years ago

OK I found a firestick4k and aspect ratio is corrected with the proposed fix regarding the video provided here https://github.com/nova-video-player/aos-AVP/issues/259 What happens is that the codec reports 1920x1088 resolution and real video as reported is 1280x544. Applying what proposes the codec results in stretched fullscreen video. It is worth noting that normally codec should report correct resolution and this is enforced on AndroidTV by proper CTS tests. But hey... amazon is almighty.

12-13 23:35:04.457   275  1056 D MtkOmxVdecEx: [0xa818d000] MtkOmxVdec::FreeBuffer all output buffers have been freed!!! signal mOutPortFreeDoneSem(1)
12-13 23:35:04.458   275  1056 D MtkOmxMVAMgr: [0xa851b5c8] Remove ion MVA Map, remain buffer count=0
12-13 23:35:04.459   275  1056 D MtkOmxVdecEx: [0xa818d000] MtkOmxVdec::SendCommand cmd=OMX_CommandPortEnable
12-13 23:35:04.459   275  6099 D MtkOmxVdecEx: [0xa818d000] # Got general command (OMX_CommandPortEnable)
12-13 23:35:04.459   275  6099 D MtkOmxVdecEx: [0xa818d000] Wait on mOutPortAllocDoneSem(0)
12-13 23:35:04.459   275  1056 D MtkOmxVdecEx: [0xa818d000] [MtkOmxVdec] Get decoder property, VDEC_DRV_QUERY_TYPE_UFO_SUPPORT: 1, 4489
12-13 23:35:04.459   275  1056 D MtkOmxVdecEx: [0xa818d000] mOutputPortDef eColorFormat(7f000001), eColorFormat(7f000001), meDecodeType(0), mForceOutputBufferCount(0), mIsUsingNativeBuffers(1)
12-13 23:35:04.459   275  1056 D MtkOmxVdecEx: [0xa818d000] [MtkOmxVdec] Get decoder property, VDEC_DRV_QUERY_TYPE_UFO_SUPPORT: 1, 4489
12-13 23:35:04.459   275  1056 D MtkOmxVdecEx: [0xa818d000] 32x32 Aligned! mOutputPortDef.nBufferSize(3133440), nStride(1920), nSliceHeight(1088) nBufferCountActual(13)
12-13 23:35:04.459   275  1056 D MtkOmxVdecEx: [0xa818d000] [MtkOmxVdec] Get decoder property, VDEC_DRV_QUERY_TYPE_UFO_SUPPORT: 1, 4489
12-13 23:35:04.459   275  1056 D MtkOmxVdecEx: [0xa818d000] mOutputPortDef eColorFormat(7f000001), eColorFormat(7f000001), meDecodeType(0), mForceOutputBufferCount(0), mIsUsingNativeBuffers(1)
12-13 23:35:04.459   275  1056 D MtkOmxVdecEx: [0xa818d000] [MtkOmxVdec] Get decoder property, VDEC_DRV_QUERY_TYPE_UFO_SUPPORT: 1, 4489
12-13 23:35:04.459   275  1056 D MtkOmxVdecEx: [0xa818d000] 32x32 Aligned! mOutputPortDef.nBufferSize(3133440), nStride(1920), nSliceHeight(1088) nBufferCountActual(13)
12-13 23:35:04.460  5974  6097 D SurfaceUtils: set up nativeWindow 0x8c26c808 for 1920x1088, color 0x7f000001, rotation 0, usage 0x2933
12-13 23:35:04.460   275  1056 E OMXNodeInstance: setParameter(1130047:MTK.DECODER.AVC, ParamPortDefinition(0x2000001)) ERROR: BadParameter(0x80001005)
12-13 23:35:04.460  5974  6097 W ACodec  : [OMX.MTK.VIDEO.DECODER.AVC] setting nBufferCountActual to 14 failed: -2147483648
12-13 23:35:04.461   275  1056 D MtkOmxVdecEx: [0xa818d000] UseBuffer:: clear mFrameBuf
12-13 23:35:04.461   275  1056 D MtkOmxMVAMgr: [0xa851b5c8] [WARNING] Can not find MVA from VA: 0xa82fc540
12-13 23:35:04.463   275  1056 D MtkOmxVdecEx: [0xa818d000] signal mOutPortAllocDoneSem (1)
12-13 23:35:04.463   275  1056 D MtkOmxVdecEx: [0xa818d000] output port populated
12-13 23:35:04.464   275  6099 D MtkOmxVdecEx: [0xa818d000] mPortReconfigInProgress as FALSE
12-13 23:35:04.495  5974  6096 I MediaCodecLogger: updateFormatChanged width = 1280 height = 544
12-13 23:35:04.499  5974  6096 I MediaCodecLogger: updatePTSTime [HW.video.avc] First PTS after Flush or reset = 14556000
12-13 23:35:04.499  5974  6096 I MediaCodecLogger: [ASAP] HW.video.avc Got First Frame Ready 434448
12-13 23:35:04.501  5974  6096 I MediaCodecLogger: [ASAP] HW.video.avc Got First Frame Render 434450
12-13 23:35:04.501  5974  6096 I MediaCodecLogger: App uses Timestamps for AVSync
12-13 23:35:04.502  5974  5995 D avos_player: init_renderer: width NOT changed: 1280 -> 1920
12-13 23:35:04.502  5974  5995 D avos_player: init_renderer: height NOT changed: 544 -> 1088
12-13 23:35:04.502  5974  5995 D avos_player: sfdec_read: INFO_FORMAT_CHANGED: 1280x544
courville commented 4 years ago

Some grep command to parse the logcat: grep -i '(avos\|gralloc\|surface\|codec\|\omx\|mali)'

courville commented 4 years ago

On Philips TV nothing strange is seen:

D avos_player: VIDEO:  [H264] [H.264] 1920x784   0fps  1964kbit/s  dec [(none)]
D avos_player: sfdec_read: INFO_FORMAT_CHANGED: 1920x784
D avos_player: sfdec_read: INFO_FORMAT_CHANGED: 1920x784
D SurfaceUtils: set up nativeWindow 0xe0019008 for 1920x784, color 0x7f000101, rotation 0, usage 0x30002900
D SurfaceController: updateSurface: lcd plugged dw=1920, dh=1080
D SurfaceController: updateSurface: lcd plugged dw=1920, dh=1080
D SurfaceController: updateSurface: vw=1920, vh=784
D SurfaceController: getVideoFormat: return mVideoFormat
D SurfaceController: updateSurface: sar=2.4489795918367347, ar=2.4489795918367347, dar=1.7777777777777777
D SurfaceController: updateSurface: VideoFormat.ORIGINAL dar<ar dh=784
D SurfaceController: updateSurface: setFixedSize(1920,784)
D SurfaceController: updateSurface: setLayoutParams(1920,784)
D SurfaceController: updateSurface: 1920x784 -> 1920x784 / formatCrop: 1.0x1.0 / mEffectMode: 0
courville commented 4 years ago

For Philips TV user reports that when plex is used or casted from PC or native Android TV player AR is ok. When using nova it is stretched with a height smaller than normal.

Capture d’écran 2019-12-14 à 10 14 38

This happens with all videos 1920x800 21:9.

courville commented 4 years ago

Full philips TV logs. novaAspectratio.txt.zip

courville commented 4 years ago

See https://github.com/nova-video-player/aos-avos/commit/e8a51498258db660ed65399ec8791a45e68814e7

courville commented 4 years ago

I will add a stretched option in the next release since it solves the wrong aspect ratio on Philips 65PUS7304 (2019 model), Philips has really made a bad implementation. I will explain, nova for a 1920x800 video displays it in a 1920x800 surface to preserve the aspect ratio. Sony does add top and bottom black bars to this video in the same surface so that it looks compressed vertically: it ends up with this video height of 592=800*800/1080. When asking the Philips TV to stretch the video to a 1920x1080 with what one could expect a really bad aspect ratio, you end up with a 1920x800 video image and a 1920x1080 surface including the black bars. I am really against messing up with the aspect ratio and thus proposing the stretched option (that some are requesting) but hey if this is the only way to get the Sony TV to behave, I will add this one. Bad implementations leads to bad compromises. Sony if you read this: boohoo!

primozcerar commented 4 years ago

I figured out what the issue was that was plaguing many apps on newer Philips TVs. It should solve the problem without having to set stretch mode on every video with AR greater than the display.

Philips Android 9 MediaCodec implementation requires that on MediaFormat used to configure the codec you set not only the KEY_WIDTH and KEY_HEIGHT but also the KEY_MAX_WIDTH and KEY_MAX_HEIGHT parameters.

This has already been tested on a Kodi build and works as expected.

courville commented 4 years ago

@primozcerar thanks for the tip. According to other references it could also fix the issue on amlogic not getting fullscreen for 4:3 video. I will investigate and it might help me remove the stretch mode.

courville commented 4 years ago

See https://github.com/xbmc/xbmc/pull/17353

courville commented 4 years ago

I noticed in xbmc that the change for KEY_MAX_WIDTH has been reverted. Are you aware of any reason for this or another way to fix the AR for philips TVs?

See https://github.com/xbmc/xbmc/pull/17557 referencing https://github.com/xbmc/xbmc/pull/17534

And the severe issues and regressions on sony and philips are described there https://github.com/xbmc/xbmc/issues/17559

primozcerar commented 4 years ago

I am not. Sorry.

V V pet., 12. jun. 2020 ob 19:09 je oseba CourvilleSoftware < notifications@github.com> napisala:

@primozcerar https://github.com/primozcerar I noticed in xbmc that the change for KEY_MAX_WIDTH has been reverted. Are you aware of any reason for this or another way to fix the AR for philips TVs?

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/nova-video-player/aos-AVP/issues/258#issuecomment-643388690, or unsubscribe https://github.com/notifications/unsubscribe-auth/ABU4RNVHPWW2LQVWYITW7VTRWJONNANCNFSM4JY5ONHQ .

courville commented 2 years ago

Closing due to inactivity and possibly prior solving. Please reopen an issue if needed.

andrei-tatar commented 2 years ago

@courville I have an issue on Google TV. When I try to set the Format to Fullscreen, it shows as stretched (H264 video file from SMB share).

If I switch to the software decoder, it all works ok. Should I stay on the software decoder?

phhusson commented 2 years ago

@andrei-tatar What's your device?

andrei-tatar commented 2 years ago

@phhusson I'm using a Google TV or Google Chromecast TV. Not sure exactly what it's called. It's the one with a small remote. image

phhusson commented 2 years ago

k so FYI the official name of this device is "Chromecast with Google TV" (I prefer calling it "sabrina" since the name is a non-sense).

@courville will test this on his own

courville commented 2 years ago

@andrei-tatar ok I confirm the issue and it seems to be a regression. Will have a look at it.

courville commented 2 years ago

@andrei-tatar quick question for you: can you please provide the video resolution of your sample and check that the video itself does not contain black top and bottom bars that would prevent the video scaling since the video aspect ratio would match the one of the screen of the TV? In this case this would not be a bug...

andrei-tatar commented 2 years ago

Sure, I can check in a few hours. But I already confirmed the software decoder works as expected. It seems to be an issue with the hardware decoder.

courville commented 2 years ago

@andrei-tatar you are right. Issue not reproduced on nvidia shield and emulator but present on all amlogic based hardware (miprod + googletv=sabrina) and only present when hwdec enabled. @phhusson do you reproduce on freePlayerPop?

EDIT: works on pop. Thus probably amlogic SDK bug. Subliminal message to Google: this should really be verified by dedicated CTS test for devices certification (sabrina does not comply).

Pentaphon commented 2 years ago

present on all amlogic based hardware

Indeed, I have a brand new Onn 4K box which has an Amlogic S905Y2 (it's an amazing value since the major bugs got fixed, btw) and I see the exact same thing as @andrei-tatar on this device when playing https://www.youtube.com/watch?v=UmBAhFSROLo which I downloaded as 1920x800 using yt-dlp. When I switch to software decoding, fullscreen and stretched look different, while original and optimized look the same.

This is interesting since the Onn 4K is just a re-brand of the ADT-3 developer box that came out in 2020. so along with the Sabrina device, that's 2 devices that Google devs can be contacted about to see if they can help with getting Amlogic to provide a fix.