VP9 stutters when hardware-decoding on M1

[xulu199705]

System and IINA version:

• macOS 14.0 beta4 23A5301h
• IINA 1.3.3 Build 138
• mpv 0.35.0-419-gf79458476b-dirty
• FFmpeg 6.0

Actual behavior:

Device: MacBook Air M1 Video: 8K, VP9

It shows frozen a few times per second, with hardware-decoding; it shows normal without hardware-decoding.

[low-batt]

Thank you for reporting this.

IINA 1.3.3 contains a workaround for issue #4486 that disables VP9 hardware decoding, but only on Intel Macs. On my M1 VP9 encoded videos are playing fine with VP9 hardware decoding enabled. This is the first report we have had of a problem under Apple Silicon.

Please post an Activity Monitor process sample of IINA when it freezes. That will confirm it is freezing the same way as was seen on Intel Macs.

Late here. I will look for a 8K video on YouTube to see if that causes a freeze for me. If you know of a YouTube video that exhibits the problem please post the link.

One possibility is that Apple has made a change to Video Toolbox for the macOS 14.0 beta that has introduced this problem. I am running macOS 13.5.

The workaround described here disables hardware decoding for just VP9, avoiding the need to turn off hardware decoding for all codecs.

[xulu199705]

My test video: https://www.youtube.com/watch?v=Mj6n0qXATRo I download the video with Downie.

---

I had this problem on IINA 1.3.2 as well. VP9 video is played non-smooth, like a ppt, but not be frozen all the time.

P.S. The sample of activity monitor is attached. IINA_sample.txt

[low-batt]

The above screen shot of the inspector window, that was taken while playing [8K HDR] 京都·无尽夏 DJI RS 3 PRO ｜Nikon Z9?

I ask because in the screen shot the value for primaries is bt.709 / bt.1886 (SDR). As is clear from the title this is supposed to be in HDR. Somehow it has been downgraded to SDR.

Did Downie change the encoding? Did you download this video recently from YouTube, or a while ago?

In trying to reproduce this I was unable to obtain an 8K version of that video encoded with VP9. Listing all the available formats from YouTube for that video using yt-dlp shows the only version available in 8K is encoded using AV1:

low-batt@gag youtube-dl$ yt-dlp -F https://www.youtube.com/watch?v=Mj6n0qXATRo
…
[info] Available formats for Mj6n0qXATRo:
ID  EXT   RESOLUTION FPS HDR CH │    FILESIZE    TBR PROTO │ VCODEC           VBR ACODEC      ABR ASR MORE INFO
──────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────
…
642 mp4   3840x2160   30 10     │ ~   1.04GiB 26548k m3u8  │ vp09.02.51.10 26548k video only
337 webm  3840x2160   30 10     │   917.33MiB 23490k https │ vp09.02.51.10 23490k video only          2160p HDR, webm_dash
702 mp4   7680x4320   30 10     │     1.72GiB 45221k https │ av01.0.16M.10 45221k video only          4320p HDR, mp4_dash

In trying many other 8K YouTube videos that said they were VP9 encoded I encountered the same thing. The maximum VP9 encoded resolution was 4K.

I'm thinking YouTube at some point made a decision to only use AV1 for 8K videos.

I downloaded the 4K VP9 encoded version of this video and it played fine with hardware encoding and was in HDR. I also downloaded the 8K AV1 version. That stuttered a bit at the start, likely due to high CPU use as hardware decoding is not available for AV1.

I spent a lot of time searching for 8K VP9 encoded videos and kept finding they had been re-encoded in HEVC. Finally I found an 8K VP9 encoded video someone had saved from quite a while ago. From the name of this video it was supposed to be HDR, but it too was in SDR.

I'm pretty sure the 8K VP9 video I found is reproducing the behavior you are reporting. When playing with hardware encoding playback stutters. With software decoding it plays fine.

For IINA 1.3.2 the libraries from mpv and FFmpeg were upgraded. The new versions of those libraries added support for VP9 hardware decoding. As discussed in issue #4486, one problem has already been submitted to the FFmpeg project.

More investigation is needed, but at this point we can conclude:

• This is not the same problem as discussed in issue #4486
• This is not due to changes in macOS 14.0 beta

[xulu199705]

The video was downloaded on June, 2022. The other HDR-titled videos download at that time are 8K VP9, also SDR. I think this is comfirmed by your inference that Google changed the encoding strategy.

And thanks for your investigations.

[singh-jagjot]

@low-batt I don't think VP9 decoding is properly supported on M1 Macs even M2 Macs for that. A few versions back MPV used to play VP9 with the SW decoder. I believe it's a bug from the MPV side that shows VP9 is decoding using the HW decoder. (I haven't run any videotoolbox VP9 checks on Xcode) Please correct me if I'm wrong.🙂

[low-batt]

Previous versions of mpv would indeed use software decoding for VP9. However that has to do with libavcodec (from FFmpeg) capabilities and how mpv makes use of the library.

Previously libavcodec did not support VP9 hardware decoding on Macs. That is why mpv was falling back to software decoding. Support was added by FFmpeg in version 6.

The mpv hwdec-codecs option controls which codecs mpv will allow libavcodec to use hardware decoding for:

--hwdec-codecs=<codec1,codec2,...|all>

Allow hardware decoding for a given list of codecs only. The special value all always allows all codecs.

You can get the list of allowed codecs with mpv --vd=help. Remove the prefix, e.g. instead of lavc:h264 use h264.

By default, this is set to h264,vc1,hevc,vp8,vp9,av1. Note that the hardware acceleration special codecs like h264_vdpau are not relevant anymore, and in fact have been removed from Libav in this form.

This is usually only needed with broken GPUs, where a codec is reported as supported, but decoding causes more problems than it solves.

By default VP9 is allowed by mpv.

If you play a VP9 encoded video in IINA with hardware decoding enabled while watching IINA's CPU use in Activity Monitor you will see CPU use increase significantly if you then disable hardware decoding.

Hardware decoding is definitely working for me on my M1 under macOS 13.5.2. Are you encountering a problem?

[singh-jagjot]

It's working for me as well on my M1 Max, CPU usage goes significantly down from hwdec=no to hwdec=auto in 4k VP9. Not sure about 8k+HDR.

[low-batt]

Glad to hear it seems to be working for you. If you encounter suspicious behavior do not hesitate to report it.

I was able to reproduce playback stuttering with 8K VP9 videos with hardware decoding enabled. I've not had time to look into that.

[low-batt]

I have reported this to FFmpeg in ticket #10963.