M1 Macs and 10bit with --vt

[LwsBtlr]

High quality 10-bit HEVC is automatically selected when using the Nvidia and Intel encoders.

Does this also apply to using the —vt flag on the M1 Macs?

[lisamelton]

@LwsBtlr No, because VideoToolbox doesn't support 10-bit HEVC output in a consistent way across macOS platforms. Currently it's not even possible for M1 Macs.

[maarut]

The latest version of ffmpeg does support 10-bit HEVC output on M1 Macs. You just need to set the pixel format to "p010le". The other options I set are ~/bin/other_video_transcoding/bin/other-transcode --dry-run --hevc --10-bit --main-audio 1=original --add-subtitle 2=forced --add-subtitle 1 --crop 3840:1608:0:276 --target 2160p=12000 --max-muxing-queue-size 1024 bluray_rip.mkv

I can't speak to the rest of the Mac lineup and what it supports.

[lisamelton]

@maarut Thanks for the heads up! I didn't realized they were finally supporting that. Currently I'm using yuv420p10le for VideoToolbox and p010le for QSV and Nvenc.

I can make a change to use p010le for all encoders in an upcoming release provided I don't run into any other issues.

[wintervaler]

I got excited this had changed based on @maarut's comment, so tried it this morning on my M1 Air. Seems that it's still impossible starting with an 8-bit source, at least on my machine -- even when I set ffmpeg to use the p010le pixel format instead of yuv420p10le. The transcode works as fine as it always has but as was the case for me when we tested this before, mediainfo reports an 8-bit output file. Is it possible you're starting with a 10-bit source? If not, does mediainfo confirm the resulting transcoded file is in fact 10-bit?

[lisamelton]

@wintervaler Ugh. Well, I was hoping the new pixel format meant good news in that regard. I really think this might be a limitation with VideoToolbox and not FFmpeg.

[lisamelton]

@wintervaler Also, thanks for testing so quickly! 👍

[maarut]

@maarut Thanks for the heads up! I didn't realized they were finally supporting that. Currently I'm using yuv420p10le for VideoToolbox and p010le for QSV and Nvenc.

Yeah, I went down in to the depths of FFMpeg to take a look. Hardware encoding with VideoToolbox doesn't support yuv420p10le on my M1 Mac which is a bit of a shame. Can't really speak to the Intel based Macs.

I can make a change to use p010le for all encoders in an upcoming release provided I don't run into any other issues.

I've made the change locally, and it appears to do what I want.

[maarut]

I got excited this had changed based on @maarut's comment, so tried it this morning on my M1 Air. Seems that it's still impossible starting with an 8-bit source, at least on my machine -- even when I set ffmpeg to use the p010le pixel format instead of yuv420p10le. The transcode works as fine as it always has but as was the case for me when we tested this before, mediainfo reports an 8-bit output file. Is it possible you're starting with a 10-bit source? If not, does mediainfo confirm the resulting transcoded file is in fact 10-bit?

I was definitely starting with a 10-bit source. How much of an advantage does converting an 8-bit source to a 10-bit output provide?

[lisamelton]

@maarut Converting an 8-bit H.264 Blu-ray rip to a much lower-bitrate 10-bit HEVC transcoding greatly reduces the risk of color banding, which is the Achilles heel of transcoding to H.264 at lower bitrates.

[wintervaler]

After some additional testing today, I have some updates! I've discovered how to convert an 8-bit source to 10-bit with FFmpeg and VideoToolbox.

The hevc_videotoolbox encoder in FFmpeg will take a -profile argument where you can specify either main (or 1) or main10 (or 2). This profile setting is automatically taken care of when you convert the pixel format using other encoders, but for some reason VideoToolbox doesn’t do that, and defaults to main.

So I specified -profile 2 (alternatively, -profile main10 also works) in the FFmpeg command and lo and behold, MediaInfo reports the output file as having 10-bit video. You can run the command with the pixel format set to yuv420p10le or p010le -- it doesn't matter. The downside here is that there doesn't appear to be a way to control the codec level with the VideoToolbox HEVC encoder, which means we're currently limited to L5.1 -- rather than the L4.0 that other-transcode normally produces with 10-bit output. As I understand it this means more player compatibility problems for any that don't support that high of a level.

This was tested on an M1 Air using a Git build of FFmpeg from Homebrew, but it should work with the 4.4 release build as well. I still get REALLY impressive performance - close to 200 fps, though it tapers off towards the end.

I think this is also perhaps relevant to issue #86.

[lisamelton]

@wintervaler Thanks for all the testing! 👍 I can definitely add this to the next version of other-transcode.

[maarut]

So I specified -profile 2 (alternatively, -profile main10 also works) in the FFmpeg command and lo and behold, MediaInfo reports the output file as having 10-bit video. You can run the command with the pixel format set to yuv420p10le or p010le -- it doesn't matter.

Be careful with setting the pixel format. FFMpeg doesn't really map yuv420p10le to a reasonable value for VideoToolbox. When I set the format to yuv420p10le for a 10-bit source, the colours in the output video came out slightly off.

[lisamelton]

@wintervaler I've checked in a fix to other-transcode implementing your suggestion. Please give it a try. Thanks.

[RusselProuse34]

@wintervaler thanks so much for testing this! I currently have an old-ish office computer in my house running my Plex and transcoding just because my Mac couldn't do 8-bit to 10-bit HEVC. If this ends up working out I'll be able to get an M1 Mac mini to do it instead, which I'm super excited about! I really appreciate you and @donmelton's hard work on this.

[lisamelton]

I just released version 0.9.0 so gem update other_video_transcoding to get all the 10-bit HEVC goodness. Thanks to all for the feedback and testing!

[RusselProuse34]

Hey Don,

Thanks again to you and the hive mind for working on this. Now that 8 to 10-bit HEVC is possible in VT, is macOS on par with Windows with a modern Nvidia GPU or Windows with an Intel QSV capable GPU for transcoding?

[lisamelton]

@RusselProuse34 You are very welcome!

All three configurations (i.e. macOS VT, Win Nvenc and Win QSV) are now functionally on par as far as 10-bit HEVC is concerned. However, you'll still probably get better quality from Nvenc. I haven't see enough output from the M1 CPU's non-QSV encoder (because I don't have one) to make that a more definite statement.

[Botts85]

@RusselProuse34 Using macOS with an Intel GPU nets very similar quality to QSV (it uses QuickSync to encode) on Windows. If you're using videotoolbox, 9th gen Intel based VTB looks better than M1 VTB.

NVENC is visibly better than both Intel and M1 based videotoolbox to me.

[lisamelton]

@Botts85 Thanks for the update!