Detect Crop on 4k videos fails because ffmpeg always reports 0:0:0:0

[eseglem]

I have recently started ripping my 4k Blu-Rays with MakeMKV and was trying to make them more portable via your awesome scripts. Unfortunately, ffmpeg is always detecting no crop, and HandBrakeCLI detects it correct. So the only time detect-crop doesn't fail is the one movie I have which actually doesn't need to be cropped.

I know that is likely an issue with ffmpeg, and not your fault. I also know I could manually input crops for each movie, but that prevents being able to just run transcode-video --crop detect on a directory of files and leaving it for a day. Is there a way to just allow HandBrakeCLI to be considered correct in these cases? With a flag or something? Or do you have any suggestions on how to automate detecting / correcting this issue?

[lisamelton]

@eseglem Congratulations on having so many 4K discs that MakeMKV can actually rip! I don't have any, which is why I haven't seen this particular failure with FFmpeg.

The good news is that you can tell transcode-video to only apply HandBrake crop values (if you trust it) by using --crop auto instead of --crop detect.

However, if you still want to leverage both HandBrake and FFmpeg for detection, you can use --crop detect --fallback-crop handbrake, and the information from FFmpeg will be ignored if they don't agree.

Does this answer your question?

[lisamelton]

@eseglem BTW, I just found a 4K video to test and it looks like FFmpeg does not return 0:0:0:0 for that one, at least.

Which version of and on which platform are you using FFmpeg?

[eseglem]

Thanks for the quick response. That is exactly what I was looking for. Think I missed it because the only mention of --fallback-crop is in the release notes from 2016, and not the main crop documentation, might be worth mentioning it up there.

Although in the mean time, I have resigned myself to probably just manually specifying them. The wrapper I am working on will probably look into a crop directory to see if a value exists, then --crop detect --fallback-crop handbrake if it doesn't.

As for 4k disks, I just picked up one of the recommended drives (ASUS BW-16D1HT) and followed the info on the MakeMKV forums, and don't have a single disc it can't rip. To include stuff that just came out today.

The main one I have been testing on is Ubuntu from the HandBrake repo, which is old. ffmpeg version 2.8.11-0ubuntu0.16.04.1 I also spun up the docker container you mention in your docs which is currently running 3.3.5 and got the same results for the couple videos I tested on. Thought looks like 3.3.5 is still a bit outdated. What version are you on? And, was the video you tested HDR, perhaps that effects it?

[lisamelton]

@eseglem No problem. And I'll consider adding that to the README document but it's way too long the way it is now. :) That's why I always recommend people look in --help output first. They can usually find the option and behavior they're looking for there.

Yes, it's much better to save your eyeball-reviewed crop values rather than rely on automatic crop detection. In fact, this is what I recommend and demonstrate here:

https://github.com/donmelton/video_transcoding#batch-control-for-transcode-video

That's great news about 4K discs! I'll have to try that one of these days. Maybe when I get a 4K TV and a third, even bigger NAS to hold the original rips. :)

I'm using FFmpeg version 3.4.2 on macOS. And that's the default/latest version installed by Homebrew.

No, I don't believe the video I tested was HDR. That could, indeed, be the culprit. But keep in mind that HandBrake is actually using LibAV, essentially the library version of FFmpeg, so that would be a bit odd.

[eseglem]

Yeah, I totally skimmed over the part about using --help, that is my bad. I was just searching the readme for info on crop.

And yeah, I really wanted to be lazy when I wrote my initial post. Then I re-read that section, for probably the third time, and finally came around to it not actually that big a deal to do it that way. Just means I need to do more re-writing of the wrapper.

Haha yeah, keeping the originals around might be rough for 4k. A couple of them are 70+GB with just the video and only the 7.1 audio track. 16 movies in 725GB. I am looking into upgrading my NAS storage right now, running out of space with all the rips, and I don't even have the originals for most. I imagine you need a TON of space.

[lisamelton]

@eseglem Sorry I took so long to reply. I went to bed early last night. :)

Yeah, I haven't pursued 4K yet due to the data storage and transcoding time requirements. Plus, I would have to repurchase close to a thousand Blu-ray Discs. :) And even then, some of those would just be expensive upscaled versions of what I have now and not really sourced from 4K masters. See this website for details if you're curious about what I mean:

http://realorfake4k.com/

Anyway, if this question is sufficiently answered, are you OK with me closing it now?

And, BTW, feel free to open other issues or mention me on Twitter (since I noticed you followed me) if you have other questions.

[eseglem]

Yeah, I went to bed too. Its all good.

I am not really planning on re-purchasing most of the Blu-rays, but I am only buying new stuff on 4k. I started buying the disks before I had a TV for them, just to future proof myself, and tha'ts still only a handful of movies. That is a very handy link so I can convince myself its not worth 'upgrading.' A few of the ones I was contemplating are 'Fake 4k' so won't bother.

I may have some more comments and questions when I actually manage to get a few more transcoded. Took 11 hours on --quick --handbrake-option encoder=x265 so need to tweak some more settings I think. And figure out 10 bit color, and if its worth it...but thats for a different issue.

Yes, this is answered totally fine, so closing it.

[lisamelton]

@eseglem Just so you know, the --quick and --veryquick options don't work with the x265 encoder. And while my special, or default, ratecontrol system does work with x265, it hasn't been tuned for that codec. You might want to try --abr, my new average bitrate ratecontrol system, with x265 instead. The ABR algorithm works the same way for x265 as it does x264.

Also, recent HandBrake nightly builds contain the x264_10bit, x265_10bit and x265_12bit encoders. While I'm not sure if they produce real HDR video, they are certainly less prone to color banding.

[eseglem]

That would explain some of it then. I just kicked something off half asleep last night to test. Is that a matter of tuning, or difference in encoders? I will play with that more as well, and look into the nightly HandBrake as well. Problem with any of that stuff is I don't actually have an HDR screen to test on. Might be better to just let it be at 8bit for now, and redo it later....but since the originals are too large to keep around right now, would have to re-rip later. I will figure it out and let you know any interesting results.

On Wed, Mar 7, 2018 at 12:33 PM, Don Melton notifications@github.com wrote:

@eseglem https://github.com/eseglem Just so you know, the --quick and --veryquick options don't work with the x265 encoder. And while my special, or default, ratecontrol system does work with x265, it hasn't been tuned for that codec. You might want to try --abr, my new average bitrate ratecontrol system, with x265 instead. The ABR algorithm works the same way for x265 as it does x264.

Also, recent HandBrake nightly builds contain the x264_10bit, x265_10bit and x265_12bit encoders. While I'm not sure if they produce real HDR video, they are certainly less prone to color banding.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/donmelton/video_transcoding/issues/198#issuecomment-371218119, or mute the thread https://github.com/notifications/unsubscribe-auth/AFpW7NkzH1txreAMhi3jyDhk6XAkNbaDks5tcBoFgaJpZM4Sfz8u .

[lisamelton]

@eseglem Thanks! Yes, please let me know.

And, yes, the tuning is due to the differences between the two encoders. My special ratecontrol system still works with x265, but I'm sure it could be improved once I have enough time to collect metrics and perform more tests. The problem is that HEVC encoding is more than twice as slow as H.264. So I'll need a significant hardware upgrade before I start.

As for --quick and --veryquick, they rely on special features and behaviors that don't exist in x265. So figuring out similar optimizations that don't compromise quality will take even longer.

Thank god I'm retired. :)