Color Banding with Dune: Part Two

[weaverm]

I've ripped the Dune: Part Two 4K UHD and I'm using two-pass-transcode.rb to transcode it down to a more portable 8-bit, h.264, non-HDR movie. I often do this with movies for which I only end up with the 4K UHD and normally it works fine. However, for this movie, the color banding in some scenes is intense.

It is pretty bad at around 32 minutes when Paul is learning to sand walk from Chani at night. It is also really bad at around 66 minutes when the Harkonnen and the Fremen are fighting in the sandy, smoky air.

My first thought was this would benefit from a 10-bit encode. However, I'm trying to keep this as basic and portable as possible since not all my playback setups have fancy AppleTVs that can play most anything. My second thought was to try re-encoding and giving it a higher bitrate. The higher bitrate versions seem to be a bit better, at the expense of file size of course. However, it seems that the libx264 encoder has a built in max bitrate of 8000kbps, which is not enough to overcome the color banding.

So, my question is what's the right thing to do here?

[lisamelton]

@weaverm I'm sorry you're having this problem. Color banding sucks and has haunted me for years. As has the extremely low contrast of Denis Villeneuve's color palette.

The bitrate target limit of 8000 Kbps for 1080p content is not built into x264. That's a feature of two-pass-transcode.

Why? Because if you can't get decent 1080p output at 8000 Kbps then it's extremely doubtful that raising the bitrate even higher will improve anything. Plus, it just sort of makes compression pointless.

You can actually bypass that limit by using the --extra option and passing a vb= argument. But don't do that, for the same reasons I just mentioned.

I hate to tell you this, but your first thought was correct. It's likely that the only way to solve a problem like this is using a 10-bit encode. But I would avoid 10-bit H.264 output since so many platforms and devices can't play that. Instead, try the hevc-transcode.rb script which uses x265.

Does that help?

[loshlee]

The newest Apple TV I have is 4th generation, but I've never been able to get it to play matroska files over Apple Home Sharing. Are you saying you have a way to make an Apple TV play .mkv over a network? Anyway, my only input would be to ask whether scaling to 1920 x 800 (or whatever matches the correct height to maintain aspect ratio for the source when the width is 1920) is a possibility for compromise. But as I said, I can't use x265. But (also), I never have color banding, either.

[lisamelton]

@loshlee Doesn't Apple TV have a Plex app? Use that. Use anything except the default media player.

[weaverm]

It seems that hevc-transcode.rb is going to preserve the HDR and 4K resolution of the source material.

It seems like I need a -x colorspace=bt709 to get back to regular blu-ray colorspace and a -x maxWidth=1920 -x maxHeight=1080 -x loose-anamorphic to get back to regular HD resolution. Does that seem right?

Do you think leaving CRF at 24 will be adequate?

@loshlee My 'main' playback setup is a Plex server with Infuse running on an AppleTV connected to a 4K TV. When I'm sitting on my couch, I can watch Dune: Part Two as a 4K rip right from the disc. It is when I'm everywhere else that I'd like something more portable and highly compatible.

[weaverm]

CRF 24 (the default) was not adequate. I also tried CRF 22 (--quality 22) and it was better, but still not great.

The 10 bit h.265 encode did significantly reduce the color banding in the problem areas I highlighted earlier.

I have tests with CRF 20 & CRF 18 encoding right now. I wonder if I'll have to switch to ABR (average bitrate) to get good results? I know other_video_transcoding used 4000 for 1080p 10-bit h.265.

[loshlee]

I'm using the Mac video toolbox, so I can get my best results for a 1080p source with a command such as this: ffmpeg -i inputfile -map 0:0 -vf crop=1920:1024:0:28 -c:v h264_videotoolbox -b:v 3000k -color_primaries:v bt709 -color_trc:v bt709 -colorspace:v bt709 -metadata:s:v "title=" -disposition:v default -map 0:1 -c:a:0 copy -metadata:s:a:0 "title=" -disposition:a:0 default -metadata:g "title=" -movflags disable_chpl outputfile

A significant variable is the 3000k bit rate. I have no idea whether that provides any useful information, but It normally results in acceptable output for me without files that are way too large. Surround sound audio (5.1) is not universally compatible, of course.

[lisamelton]

@weaverm Yes, Adding -x maxWidth=1920 -x maxHeight=1080 -x loose-anamorphic -x colorspace=bt709 to your command line will mimic the behavior of two-pass-transcode.rb when dealing with 4K HDR inputs.

You can certainly try ABR ratecontrol with hevc-transcode.rb using the -b option. However, I've had very mixed results with that for grainy or very dynamic inputs. The good news is that particular movie is neither.

As for target bitrate, 4000 Kbps should be more than enough.

[loshlee]

nlmeans can help with graininess for ffmpeg. I'm assuming that's also available for Handbrake and more video transcoding (-x nlmeans=4.65:0.85:7:3:3:1 for example).

[lisamelton]

@loshlee Yes, it's available for HandBrake. But my advice is to never use it.

[weaverm]

I found that CRF 20 was still inadequate. CRF 18 was barely acceptable. CRF 16 was a reasonably good encode. The 10-bit h.265 file was 3.89 GB according to Finder.

I also tried, in the name of science, an 8-bit, 2-pass, h.264 encode with the bitrate forced to 10,000. It produced a giant, 13.24 GB file, nearly 2x the size of the file with the default settings (7 GB). That makes sense since the bitrate was set to 2x the default. The result was a pretty decent encode. Most of the troublesome color banding was gone and there seemed to be no blockiness or other visual distractions. The 10K bitrate encode is definitely an improvement over the 8K bitrate encode.

I'm also going to try a few variations of 10-bit h.265 ABR. Fortunately, my machine that does these encodes doesn't complain. I've done more than a dozen encodes of this movie in the last couple of days. :)

[loshlee]

OK, I see that you have a Mac. if it's not too much of an imposition on your privacy, you may want to have a look at the AppleScript gist I just created that allows you to get general media-info in Terminal from a dropped media file. You should be able to copy it and paste it into Script Editor to save it as an application. Please read the comment for suggestions about how to use it. Then you can add pertinent information from Media-info here so we may be able to help more. I find it useful to have around anyway, and you may wish to offer suggestions about how I might improve it or make it more useful.

[weaverm]

@loshlee Which files would you like mediainfo from?

[weaverm]

I've encoded several average bitrate versions of the movie using hevc-transcode.rb as well. To my eye, the 4000kbps 10-bit h.265 looks the best; even better than the CRF 16 version. Seemingly the most difficult moment of the movie is at 1:07:30, where Muad'Dib walks out of the smoke. I also tried a 5000kbps 10-bit h.265 version and at that moment, it maybe looks a tiny bit better, but I don't know if I'd be able to tell the difference in a blind test.

In the end, my conclusion is as @lisamelton predicted. A 10-bit, h.265, average bitrate 4000 encode produces pretty good results for this movie. If you really want an 8-bit h.264 version, then with a 2 pass encode the average bitrate needs to pushed to a ridiculous 10,000.

[weaverm]

Thanks for your help.

[lisamelton]

@weaverm You are very welcome! 💖

[loshlee]

@loshlee Which files would you like mediainfo from?

Just the source (ripped). But sounds like it’s moot now for this thread, which is good.