Error transcoding to HEVC on a 2019 MacBook Pro

[RusselProuse34]

Hi all,

I'm trying to transcode some h.264 video to HEVC on my 2019 MacBook Pro and other-transcode is giving me an error. By default it will select the slower software based x265 tool to complete the task, and if I specify hardware encoding with the --vt flag, it gives me an error. Below is the output from Terminal:

other-transcode --hevc --mp4 --vt /Users/russel/Desktop/Kim\'s\ Convenience\ -\ S04E01.mkv Verifying "ffprobe" availability... Verifying "ffmpeg" availability... Verifying "mkvpropedit" availability... Finding encoders... Scanning media... duration = 00:21:53.12 Stream mapping: 0 = hevc_videotoolbox / 4000 Kbps 1 = copy Command line: ffmpeg -loglevel error -stats -i /Users/russel/Desktop/Kim\'s\ Convenience\ -\ S04E01.mkv -map 0:0 -c:v hevc_videotoolbox -b:v 4000k -color_primaries:v bt709 -color_trc:v bt709 -colorspace:v bt709 -metadata:s:v title\= -disposition:v default -tag:v hvc1 -map 0:1 -c:a:0 copy -metadata:s:a:0 title\= -disposition:a:0 default -sn -metadata:g title\= -movflags disable_chpl Kim\'s\ Convenience\ -\ S04E01.mp4 Transcoding... [hevc_videotoolbox @ 0x7ff5dd808e00] Error encoding frame: -12905 [hevc_videotoolbox @ 0x7ff5dd808e00] popping: -542398533 Error initializing output stream 0:0 -- Error while opening encoder for output stream #0:0 - maybe incorrect parameters such as bit_rate, rate, width or height /usr/local/bin/other-transcode: transcoding failed: Kim's Convenience - S04E01.mp4

Any idea what I could be doing wrong?

[lisamelton]

@RusselProuse34 I'm sorry you're having this problem.

Are you able to access the --vt encoder if you don't add --hevc to your command line? IOW, can you create H.264 output using VideoToolbox?

[RusselProuse34]

Thanks for the quick response @donmelton. Yes, if I leave out —hevc, the system uses VideoToolbox automatically whether I add the —vt argument. So it seems to just be a problem with transcoding to HEVC.

[lisamelton]

@RusselProuse34 Thanks for clarifying that! Just wanted to make sure.

OK, I checked online and 2019-vintage MacBook Pros should be equipped with a HEVC encoder in hardware. I believe it's either in the T1 chip or on the video card and not in the Intel CPU. Is there any odd power-saving mode you've enabled on your machine? That's really the only thing I can think of because my 2018 MacBook Pro can do HEVC in hardware.

[RusselProuse34]

@donmelton I've had a look and confirmed that my Mac isn't running in Low Power Mode on either battery or while plugged in.

I'm going to uninstall Homebrew and the associated packages I set up to use with other_video_transcoding and reinstall them to see if that helps. I'll report back if anything changes.

[RusselProuse34]

@donmelton no dice, same error :(

Is there anything you can think of that I can try, or any logs I can post to help diagnose the problem?

[lisamelton]

@RusselProuse34 Which version of ffmpeg are you using and where did you get it? And including a .log file could help.

[RusselProuse34]

@donmelton ffmpeg version 4.4.1. I ran ffmpeg -version and it gave me a bunch of other info, so I can post that too if it would help.

I've attached a log file as well. _ffmpeg_27442_64736.mkv.log

[lisamelton]

@RusselProuse34 OK, looking at your .log file it looks like you're trying to transcode an previously transcoded file. And there might be something odd about that file. So, do you have a standard Blu-ray rip handy to try instead?

[RusselProuse34]

@donmelton I'll have a look and try some different files. Will report back shortly.

[RusselProuse34]

@donmelton I don't have any un-transcoded Blu-Ray rips on hand but I have some stuff I had purchased from iTunes downloaded onto my machine. Not sure if there's DRM or anything that prevents it from being transcoded, but it threw a different error this time. Log file attached.

other-transcode --hevc --vt /Users/russel/Desktop/01\ Season\ 1\,\ Episode\ 1_\ Myanmar\ \(1080p\ HD\).m4v Verifying "ffprobe" availability... Verifying "ffmpeg" availability... Verifying "mkvpropedit" availability... Finding encoders... Scanning media... duration = 00:42:07.486667 Stream mapping: 1 = hevc_videotoolbox / 4000 Kbps 0 = copy Command line: ffmpeg -loglevel error -stats -i /Users/russel/Desktop/01\ Season\ 1,\ Episode\ 1_\ Myanmar\ \(1080p\ HD\).m4v -map 0:1 -c:v hevc_videotoolbox -b:v 4000k -color_primaries:v bt709 -color_trc:v bt709 -colorspace:v bt709 -metadata:s:v title\= -disposition:v default -map 0:0 -c:a:0 copy -metadata:s:a:0 title\= -disposition:a:0 default -sn -metadata:g title\= -default_mode passthrough 01\ Season\ 1,\ Episode\ 1_\ Myanmar\ \(1080p\ HD\).mkv Transcoding... Decoder (codec none) not found for input stream #0:1 /usr/local/bin/other-transcode: transcoding failed: 01 Season 1, Episode 1_ Myanmar (1080p HD).mkv _ffmpeg_21622_65952.mkv.log

[lisamelton]

@RusselProuse34 Yeah, that's a DRM problem.

[RusselProuse34]

@donmelton OK makes sense.

I tried with a few other files that I have that don't have DRM and I'm getting the same error as I was originally, down to the exact frame. Unfortunately the files aren't original rips, but I've been transcoding non-original rips on some other machines for a long time without issue.

Happy to include further logs or try anything else you may find helpful.

[lisamelton]

@RusselProuse34 At this point I'm stumped as to what's going on. I've even contacted the HiveMind™ and they're all scratching their heads too. I am truly sorry I can't be more helpful yet.

[RusselProuse34]

@donmelton No problem! I appreciate the support. It's great that you've got such a large community of experts that can help with issues of all kinds. Let me know if I can be of more assistance to you or anyone else in the HiveMind.

[douglsmith]

I also have a 2019 MacBook Pro. A year ago I know that I was getting hardware encoding of hevc because I kept notes from then. Now it is going to the software encoder. The changes I know of would be a newer version of macOS and everything updated to current through homebrew.

I am also getting an error message if I try to force hardware hevc. For example, I tried a rip from my Blu-ray of Hugo: other-transcode --mp4 --vt --hevc Hugo_t00.mkv

_ffmpeg_33540_10713.mp4.log

ffmpeg -version
ffmpeg version 4.4.1 Copyright (c) 2000-2021 the FFmpeg developers
built with Apple clang version 13.0.0 (clang-1300.0.29.3)
configuration: --prefix=/usr/local/Cellar/ffmpeg/4.4.1_3 --enable-shared --enable-pthreads --enable-version3 --cc=clang --host-cflags= --host-ldflags= --enable-ffplay --enable-gnutls --enable-gpl --enable-libaom --enable-libbluray --enable-libdav1d --enable-libmp3lame --enable-libopus --enable-librav1e --enable-librist --enable-librubberband --enable-libsnappy --enable-libsrt --enable-libtesseract --enable-libtheora --enable-libvidstab --enable-libvmaf --enable-libvorbis --enable-libvpx --enable-libwebp --enable-libx264 --enable-libx265 --enable-libxml2 --enable-libxvid --enable-lzma --enable-libfontconfig --enable-libfreetype --enable-frei0r --enable-libass --enable-libopencore-amrnb --enable-libopencore-amrwb --enable-libopenjpeg --enable-libspeex --enable-libsoxr --enable-libzmq --enable-libzimg --disable-libjack --disable-indev=jack --enable-avresample --enable-videotoolbox
libavutil      56. 70.100 / 56. 70.100
libavcodec     58.134.100 / 58.134.100
libavformat    58. 76.100 / 58. 76.100
libavdevice    58. 13.100 / 58. 13.100
libavfilter     7.110.100 /  7.110.100
libavresample   4.  0.  0 /  4.  0.  0
libswscale      5.  9.100 /  5.  9.100
libswresample   3.  9.100 /  3.  9.100
libpostproc    55.  9.100 / 55.  9.100

Anything else I can provide that might help?

[lisamelton]

@douglsmith I'm sorry you're having this problem. I'm starting to think this is something that changed in Apple System Software and not FFmpeg. It's certainly nothing that changed in other-transcode. So far none of my usual suspects (other nerds) have figured this out either.

[khaosx]

@douglsmith Can you give us the text output from mediainfo for your source file?

[skj-dev]

@douglsmith Unrelated to the rest, you might want to remove that system report. There's a lot of information in there that you might not want to have hanging out in a public space.

[douglsmith]

@khaosx Here is the mediainfo output: Hugo-mediainfo.txt

@ttyS0 Thanks for that.

[lisamelton]

@douglsmith Thanks! I don't see anything unusual in that MediaInfo output though.

[douglsmith]

I installed macOS Big Sur on an external boot drive and did a bunch of testing comparing it with macOS Monterey on my 2019 MacBook Pro. The results were quite interesting.

TL;DR Some commands work in Big Sur that no longer work in Monterey, but there is a possible workaround.

I started with a test video that is a 01:44 portion of a movie ripped from a Blu-ray. It is 8-bit according to mediainfo. For each test I recorded the encoding time, the file size, and the bit depth since those are good indicators of the kind of encoding that actually happened.

These parameters: other-transcode --crop auto --mp4 --hevc testvideo.mkv were successful under Monterey. It executed this command, which did software encoding: ffmpeg -loglevel error -stats -i testvideo.mkv -map 0:0 -c:v libx265 -pix_fmt:v yuv420p10le -b:v 4000k -maxrate:v 12000k -bufsize:v 12000k -color_primaries:v bt709 -color_trc:v bt709 -colorspace:v bt709 -metadata:s:v title\= -disposition:v default -tag:v hvc1 -map 0:1 -c:a:0 ac3 -b:a:0 640k -metadata:s:a:0 title\= -disposition:a:0 default -sn -metadata:g title\= -movflags disable_chpl testvideo.mp4 Elapsed time: 00:02:13, bytes: 54,893,775, 10 bits

The same parameters were successful under Big Sur but the command it executed was different and used hardware encoding: ffmpeg -loglevel error -stats -i testvideo.mkv -map 0:0 -c:v hevc_videotoolbox -b:v 4000k -color_primaries:v bt709 -color_trc:v bt709 -colorspace:v bt709 -metadata:s:v title\= -disposition:v default -tag:v hvc1 -map 0:1 -c:a:0 ac3 -metadata:s:a:0 title\= -disposition:a:0 default -sn -metadata:g title\= -movflags disable_chpl testvideo.mp4 Elapsed time: 00:00:18, bytes: 57,759,781, 8 bits

So under Monterey it used the software encoder at 10 bits but under Big Sur it used the hardware encoder at 8 bits.

Then I tried forcing the hardware encoder: other-transcode --crop auto --mp4 --hevc --vt testvideo.mkv Under Monterey it crashed and burned: ffmpeg -loglevel error -stats -i testvideo.mkv -map 0:0 -c:v hevc_videotoolbox -b:v 4000k -color_primaries:v bt709 -color_trc:v bt709 -colorspace:v bt709 -metadata:s:v title\= -disposition:v default -tag:v hvc1 -map 0:1 -c:a:0 ac3 -b:a:0 640k -metadata:s:a:0 title\= -disposition:a:0 default -sn -metadata:g title\= -movflags disable_chpl testvideo.mp4

Transcoding...
[hevc_videotoolbox @ 0x7facd086b000] Error encoding frame: -12905
[hevc_videotoolbox @ 0x7facd086b000] popping: -542398533
Error initializing output stream 0:0 -- Error while opening encoder for output stream #0:0 - maybe incorrect parameters such as bit_rate, rate, width or height
/usr/local/bin/other-transcode: transcoding failed: testvideo.mp4

But it ran successfully under Big Sur: ffmpeg -loglevel error -stats -i testvideo.mkv -map 0:0 -c:v hevc_videotoolbox -b:v 4000k -color_primaries:v bt709 -color_trc:v bt709 -colorspace:v bt709 -metadata:s:v title\= -disposition:v default -tag:v hvc1 -map 0:1 -c:a:0 ac3 -metadata:s:a:0 title\= -disposition:a:0 default -sn -metadata:g title\= -movflags disable_chpl testvideo.mp4 Elapsed time: 00:00:18, bytes: 57,759,781, 10 bits (mediainfo)

The ffmpeg command line was slightly different for each OS and caused ffmpeg to error on Monterey. It says it's 10 bits under Big Sur. The files size is the exact same number of bytes as the previous test in Big Sur that says it's 8 bits. Can that be true or is the metadata wrong?

Next I tried forcing both the hardware encoder and 10-bit: other-transcode --crop auto --mp4 --hevc --vt --10-bit testvideo.mkv It actually completed without error on Monterey by adding --10-bit: ffmpeg -loglevel error -stats -i testvideo.mkv -map 0:0 -c:v hevc_videotoolbox -pix_fmt:v yuv420p10le -b:v 4000k -color_primaries:v bt709 -color_trc:v bt709 -colorspace:v bt709 -metadata:s:v title\= -disposition:v default -tag:v hvc1 -map 0:1 -c:a:0 ac3 -b:a:0 640k -metadata:s:a:0 title\= -disposition:a:0 default -sn -metadata:g title\= -movflags disable_chpl testvideo.mp4 Elapsed time: 00:00:36, bytes: 60,224,739, 10 bits (mediainfo)

The short encode time tells me that it was actually hardware encoding as the ffmpeg command shows. This may be a viable workaround. I've tested it many times now with other files and it is working consistently.

The same parameters under Big Sur also worked and produced this: ffmpeg -loglevel error -stats -i testvideo.mkv -map 0:0 -c:v hevc_videotoolbox -pix_fmt:v p010le -b:v 4000k -profile:v main10 -color_primaries:v bt709 -color_trc:v bt709 -colorspace:v bt709 -metadata:s:v title\= -disposition:v default -tag:v hvc1 -map 0:1 -c:a:0 ac3 -metadata:s:a:0 title\= -disposition:a:0 default -sn -metadata:g title\= -movflags disable_chpl testvideo.mp4 Elapsed time: 00:00:33, bytes: 57,737,187, 10 bits (mediainfo)

Is it Other Transcode that is generating the different ffmpeg command line under each operating system when starting with the same parameters?

I hope that is useful in some way.

[lisamelton]

@douglsmith You have definitely went the extra mile in testing, sir. Well done and thanks for the additional information!

Yes, the other-transcode tool is generating the ffmpeg command. That's essentially most of what other-transcode does, anyway.

The question is why is the command being generated different under two different versions of macOS? And why an explicit --10-bit option seems to fix things?

You're bypassing my dynamic hardware encoder detection code when you explicitly use--vt on the command line. And I don't force 10-bit for the VideoToolbox HEVC encoder like I do for the Nvenc, QSV and x265 HEVC encoders.

Sooooo... there's something weird going on here and I still don't know what it is. Because I don't have this problem with my Intel MacBook Pro of a similar vintage.

[douglsmith]

@donmelton Let me know if there's anything else I can test here. I'm happy to help.

BTW, I'll probably run the same tests on an M1 MacBook Pro when I get a chance just because I'm curious.

[douglsmith]

@donmelton What do you think about these two commands from the above tests under Big Sur generating files of the exact same size in bytes but mediainfo says the first has a bit depth of 8 bits and the second 10 bits? Is that possible?

other-transcode --crop auto --mp4 --hevc testvideo.mkv other-transcode --crop auto --mp4 --hevc --vt testvideo.mkv

The only difference I see in the ffmpeg command line is that the second version adds -b:a:0 640k. Both used hardware encoding.

[lisamelton]

@douglsmith That makes no sense that the audio options would change. What version of other-transcode are you using?

[douglsmith]

Aha! I've been using version 0.7.0. I forgot this wasn't updated with brew like many of the other utilities I use and keep up to date. When I made by Big Sur boot drive I installed all fresh copies of everything so that would have been version 0.9.0, which explains the differences. I've now updated to 0.9.0 and I'll try again.

[lisamelton]

@douglsmith LOL! That explains a LOT then. :) No worries. Let me know what you find out.

[douglsmith]

Okay, I'm sure everything is up to date with other-transcode 0.9.0 now. I also double-checked that when I made my Big Sur boot volume I had installed version 0.9.0 there too. All good.

other-transcode --crop auto --mp4 --hevc testvideo.mkv This works as before and uses the software encoder on Monterey but the hardware encoder on Big Sur. On Monterey, besides -c:v hevc_videotoolbox, it also adds -pix_fmt:v yuv420p10le, -maxrate:v 12000k, and -bufsize:v 12000k. So we still have differences per operating system for some reason.

other-transcode --crop auto --mp4 --hevc --vt testvideo.mkv Attempting to force hardware encoding with --vt resulted in the same error as before. The ffmpeg command line generated is exactly the same on macOS Monterey and Big Sur but for some reason it works on Big Sur and fails on Monterey.

other-transcode --crop auto --mp4 --hevc --vt --10-bit testvideo.mkv Trying the --10-bit parameter still makes it work. The ffmpeg command line generated is exactly the same on macOS Monterey and Big Sur.

[lisamelton]

@douglsmith Thanks for that excellent and useful synopsis!

I have no idea why it would behave differently on Monterey and Big Sur. This is either an issue with the OS or with ffmpeg since the explicit options bypass my detection code.

For now, the workaround seems to be that you need to add the --10-bit option.

[douglsmith]

Tagging the original poster, @RusselProuse34. See the workaround above.

[lisamelton]

@douglsmith Good idea and thanks! I hope it works for him as well.

[RusselProuse34]

@douglsmith Thanks so much for all of your testing on this! I transcoded a bunch of files last night, and adding the --10-bit flag did indeed trigger the hardware transcoder on my machine in Monterey. Super strange why the outputs would be different depending on OS and without that flag, but it's nice to know that there's a workaround. Thanks again!

@donmelton You're welcome to close this if you'd like - I'm not sure the problem is exactly resolved, but now that we have a workaround things are solved for me.

[lisamelton]

@RusselProuse34 Thanks! I'll close this for now because I can't really force --10-bit for VideoToolbox like I do the the Nvidia encoders. So, we'll just live with the workaround for now.