Intelligent switching between 1080p and 4K

[diamondsw]

When I first installed the beta of Aerial for Mojave and tvOS 12 screen saver support, I was greeted with slideshow performance on my 3-display 2013 MacBook Pro. This all cleared up when I switched from the new 4K default back to 1080p - my poor MacBook with integrated graphics can't push that much video apparently. However, the only reason I knew that toggle was there was because I'd been reading a few issue threads in here. I imagine a lot of people won't know this, and the default to 4K will not serve them well. All they know is performance tanked.

I'd request an "Auto" setting to dynamically choose 1080p videos for any screen 1920px wide (or less), and 4K videos for screens above that resolution. No sense in pushing 4K video where it's not visible. If possible, handle this on a monitor-by-monitor basis.

[glouel]

The first beta only had 4K support, then I added back 1080p. Because of that I kept it as default to avoid people redownloading videos.

However I definitely agree, 1080p should probably even be the default for everyone (the whole 4K set takes 12.5 GB which may be an issue for a lot of people by itself).

Side note, the new videos (1080p and 4K) are all using HEVC instead of H264. H.264 is still available for 1080p but I haven't added the option to use them yet. HEVC has a pretty big size advantage. On older configurations, HEVC may be an issue. In your specific case, I guess you have a dedicated GPU on your MBP (that may handle 1080p HEVC but not 4K or not 3x 4K streams). Haswell's iGPU doesn't hw decode HEVC at all, so that's probably an issue that needs fixing too for those machines and older.

Ideally, we should probably detect if HEVC decoding is hardware accelerated or not too. In the case of Intel's iGPU, full HEVC decode starts with Skylake (2015).

[eivindml]

Is it only 1080p at this point? It looks really grainy on my LG 5K display. I wouldn't have a problem with using 12GB.

[glouel]

If you are using my builds from the other thread, check in the cache panel if you are using 4K. Some videos are grainy/blurry (especially space), some aren't. But if it's 4K, it should only use 4K. If you want to compare, you can go manually in the cache folder and check the videos. All the new ones you have downloaded should be ending in something like "HEVC_4K.mov" (you may also have some previous videos in that folder but they won't be used). 1080p shows as "HEVC 2K".

[diamondsw]

The 13" MacBook Pro only had integrated graphics, and being as old as it is I'm pretty sure I don't have HEVC capability. Screen savers display fine on 1080p for me, including the lovely new ISS ones. Oddly, previewing even 4K HEVC files in Quicklook is buttery smooth, although that is one video at a time.

Here's my Aerial movie cache: b1-1.mov b1-2.mov b1-3.mov b1-4.mov b2-1.mov b2-2.mov b2-3.mov b2-4.mov b3-1.mov b3-2.mov b3-3.mov b4-1.mov b4-2.mov b4-3.mov b5-1.mov b5-2.mov b5-3.mov b6-1.mov b6-2.mov b6-3.mov b6-4.mov b7-1.mov b7-2.mov b7-3.mov b8-1.mov b8-2.mov b8-3.mov b9-3.mov b10-3.mov comp_A001_C004_1207W5_v23_SDR_FINAL_20180706_SDR_2K_HEVC.mov comp_A001_C004_1207W5_v23_SDR_FINAL_20180706_SDR_4K_HEVC.mov comp_A009_C001_010181A_v09_SDR_PS_FINAL_20180725_SDR_2K_HEVC.mov comp_A009_C001_010181A_v09_SDR_PS_FINAL_20180725_SDR_4K_HEVC.mov comp_A083_C002_1130KZ_v04_SDR_PS_FINAL_20180725_SDR_2K_HEVC.mov comp_A083_C002_1130KZ_v04_SDR_PS_FINAL_20180725_SDR_4K_HEVC.mov comp_A103_C002_0205DG_v12_SDR_FINAL_20180706_SDR_4K_HEVC.mov comp_A105_C003_0212CT_FLARE_v10_SDR_PS_FINAL_20180711_SDR_4K_HEVC.mov comp_A108_C001_v09_SDR_FINAL_22062018_SDR_4K_HEVC.mov comp_A114_C001_0305OT_v10_SDR_FINAL_22062018_SDR_2K_HEVC.mov comp_A114_C001_0305OT_v10_SDR_FINAL_22062018_SDR_4K_HEVC.mov comp_C002_C005_0818SC_001_v01_6M_HB_tag0.mov comp_C007_C004_0824AJ_001_v01_6M_HB_tag0.mov comp_C007_C011_08244D_001_v01_6M_HB_tag0.mov comp_CH_C002_C005_PSNK_v05_SDR_PS_FINAL_20180709_SDR_2K_HEVC.mov comp_CH_C002_C005_PSNK_v05_SDR_PS_FINAL_20180709_SDR_4K_HEVC.mov comp_CH_C007_C004_PSNK_v02_SDR_PS_FINAL_20180709_SDR_4K_HEVC.mov comp_CH_C007_C011_PSNK_v02_SDR_PS_FINAL_20180709_SDR_2K_HEVC.mov comp_CH_C007_C011_PSNK_v02_SDR_PS_FINAL_20180709_SDR_4K_HEVC.mov comp_DB_D001_C001_v03_6Mbps.mov comp_DB_D001_C005_t9_6M_HB_tag0.mov comp_DB_D002_C003_t9_6M_HB_tag0.mov comp_DB_D008_C010_v04_6Mbps.mov comp_DB_D011_C010_v10_6Mbps.mov comp_DB_D011_D009_SIGNCMP_v15_6Mbps.mov comp_GL_G004_C010_v03_6Mbps.mov comp_GL_G010_C006_v08_6Mbps.mov comp_GMT026_363A_103NC_E1027_KOREA_JAPAN_NIGHT_v17_SDR_FINAL_25062018_SDR_2K_HEVC.mov comp_GMT026_363A_103NC_E1027_KOREA_JAPAN_NIGHT_v17_SDR_FINAL_25062018_SDR_4K_HEVC.mov comp_GMT306_139NC_139J_3066_CALI_TO_VEGAS_v07_SDR_FINAL_22062018_SDR_2K_HEVC.mov comp_GMT306_139NC_139J_3066_CALI_TO_VEGAS_v07_SDR_FINAL_22062018_SDR_4K_HEVC.mov comp_GMT308_139K_142NC_CARIBBEAN_DAY_v09_SDR_FINAL_22062018_SDR_4K_HEVC.mov comp_GMT312_162NC_139M_1041_AFRICA_NIGHT_v14_SDR_FINAL_20180706_SDR_4K_HEVC.mov comp_GMT329_113NC_396B_1105_CHINA_v04_SDR_FINAL_20180706_F900F2700_SDR_4K_HEVC.mov comp_GMT329_117NC_401C_1037_IRELAND_TO_ASIA_v48_SDR_PS_FINAL_20180725_F0F6300_SDR_4K_HEVC.mov comp_HK_B005_C011_t9_6M_tag0.mov comp_HK_H004_C008_v10_6Mbps.mov comp_HK_H004_C010_4k_v01_6Mbps.mov comp_HK_H004_C013_t9_6M_HB_tag0.mov comp_LA_A005_C009_PSNK_ALT_v08_SDR_PS_FINAL_20180709_SDR_2K_HEVC.mov comp_LA_A005_C009_PSNK_ALT_v08_SDR_PS_FINAL_20180709_SDR_4K_HEVC.mov comp_LA_A005_C009_v05_t9_6M.mov comp_LA_A006_C004_v01_SDR_FINAL_PS_20180730_SDR_4K_HEVC.mov comp_LA_A006_C008_t9_6M_HB_tag0.mov comp_LA_A008_C004_ALT_v33_6Mbps.mov comp_LA_A009_C009_PSNK_v02_SDR_PS_FINAL_20180709_SDR_4K_HEVC.mov comp_LA_A009_C009_t9_6M_tag0.mov comp_LW_L001_C006_t9_6M_tag0.mov DB_D001_C001_2K_SDR_HEVC.mov DB_D001_C001_4K_SDR_HEVC.mov DB_D001_C005_4K_SDR_HEVC.mov DB_D002_C003_2K_SDR_HEVC.mov DB_D002_C003_4K_SDR_HEVC.mov DB_D008_C010_4K_SDR_HEVC.mov DB_D011_C010_4K_SDR_HEVC.mov GL_G002_C002_2K_SDR_HEVC.mov GL_G002_C002_4K_SDR_HEVC.mov GL_G004_C010_4K_SDR_HEVC.mov HK_B005_C011_2K_SDR_HEVC.mov HK_B005_C011_4K_SDR_HEVC.mov HK_H004_C001_2K_SDR_HEVC.mov HK_H004_C001_4K_SDR_HEVC.mov HK_H004_C008_2K_SDR_HEVC.mov HK_H004_C008_4K_SDR_HEVC.mov HK_H004_C010_2K_SDR_HEVC.mov HK_H004_C010_4K_SDR_HEVC.mov HK_H004_C013_2K_SDR_HEVC.mov HK_H004_C013_4K_SDR_HEVC.mov LA_A006_C008_2K_SDR_HEVC.mov LA_A006_C008_4K_SDR_HEVC.mov LA_A008_C004_2K_SDR_HEVC.mov LA_A008_C004_4K_SDR_HEVC.mov plate_G002_C002_BG_t9_6M_HB_tag0.mov

[diamondsw]

I should also note, checking the horizontal resolution is probably a better idea than vertical, owing to so many 16x10 monitors out there. Otherwise you'll scale up to 4K needlessly on the common 1920x1200 resolution.

[glouel]

Well, HEVC should still play on your Mac but it will be CPU decoded instead of GPU accelerated, so you may see high CPU load (see this issue https://github.com/JohnCoates/Aerial/issues/505 )

I've added a build here with 1080p H264 as default : https://github.com/JohnCoates/Aerial/issues/463#issuecomment-424407877

We'll see what we can do about detection at a later point.

[BreiteSeite]

I also have moderate cpu and high gpu load on my 2016 MacBook TouchBar with the Radeon Pro 455 connected to the LG UltraFine 5k display on macOS 10.14. It ramps up the fan very noticeably. It sometimes even drops frame.

I cached all videos in 4k HEVC.

Shouldn't it be very smooth and non-taxin, due to hardware HEVC decoding support?

[glouel]

@BreiteSeite I tried to investigate that a bit in that issue https://github.com/JohnCoates/Aerial/issues/512#issuecomment-426052836

In your case, Late 2016, that's still a Skylake CPU so you only have "semi" accelerated HEVC decoding, because the videos are encoded in Main10 HEVC profile.

As for your Radeon Pro 455, in theory it should have HEVC 10 bit playback, whether it is used or not, I don't know. It may not be used at all, or because you have two gpus, only some of the time. I'd suggest as I did in the linked comment checking CPU usage when playing the same video on QuickTime, see what happens ?

[BreiteSeite]

@glouel 1st: thank you for the awesome and really quick issue support/reaction!

I already tried playing the video via quicktime. And in activity monitor, you can really see how taxing it is to the GPU.

This is with the 5k display connected, which forces the discrete gpu anyway.

[glouel]

@BreiteSeite I'm not sure that 100% means it uses the GPU for decoding. The decoding is based on fixed function units, not on the GPU itself so it's hard to say what that GPU load is (I'm not sure what sensor is used). Note that the GPU itself doesn't have to be active in order for Intel's QuickSync to be used so forcing the discrete GPU may not affect MacOS's GVA video decoding.

In any case, you still have a fair bit of CPU load. I'd suggest you check with Activity Monitor, see what CPU load you have on VTDecoderXPCService with QuickTime, that would give us a good clue.

[BreiteSeite]

@glouel

[glouel]

@BreiteSeite Thanks !

So I'd say it's definitely not fully GPU accelerated, and probably not playing through your Polaris UVD. I guess it uses Intel's QuickSync by default, your CPU load looks close to what I have on my iMac which also lacks Main10 decoding on QuickSync (and on my GPU).

I did check the GPU load thing, and as I suspected, you see a "GPU load" just from the video playback itself (I guess from the buffers). Here's what I have with QuickSync "assisted" (but not complete) HEVC decoding :

If you try a 1080p H.264 video you'll probably see VTDecoderXPCService below 5%. Or the same thing with a HEVC Main (8bit) video.

Ideally for those machines (2015/2016), reencoding to 8bit would fix the issue. Bit of a mess !

[glouel]

Closing as fixed in release 1.4, we're still working on better detection in this issue : https://github.com/JohnCoates/Aerial/issues/522