lisamelton / video_transcoding

Tools to transcode, inspect and convert videos.
MIT License
2.39k stars 160 forks source link

A little explanation on the --quick agrument #182

Closed steveowashere closed 6 years ago

steveowashere commented 6 years ago

Hello, I'm just curious about the details of the --quick argument. I've been using it a few times and loving the speed increase, but re-reading the documentation, I noticed that:

The --quick option avoids the typical quality problems associated with the x264 video encoder's speed-based presets

and

you can increase encoding speed by 70-80% with no easily perceptible loss in video quality

I'm just wondering if this can be elaborated on a little bit? What are the quality problems with the speed-based presets? The parts about no perceptible loss of quality, I'm just curious what is sacrificed (if anything) and is there a situation where --quick shouldn't be used.

I generally use transcode-video on my blu-ray rips and I like to focus on quality over size (within reason, 15gb max for a 2 hour movie is my limit), so what options would be recommend in this case?

lisamelton commented 6 years ago

@steveowashere That's a great question! I really should write a blog post about this sometime.

I'm just waking up now so I want to respond briefly but quickly to let you know that a longer answer is coming. But I need to get some coffee and other business taken care of today before I write what could be a novel-length post here in GitHub. :)

Seriously, expect something later tonight or tomorrow morning. Thanks!

steveowashere commented 6 years ago

Thanks very much, look forward to reading what you have to say 👍

lisamelton commented 6 years ago

@steveowashere OK, here is my much longer answer...

First, let's talk about those "quality problems associated with x264 presets."

The x264 encoder has five presets (fast, faster, veryfast, superfast and ultrafast) in escalating order to speed up the encoding process. And they all behave pretty much as their names indicate, achieving incremental increases in performance by degrading accuracy and compression efficiency in small steps.

Each preset does this by changing various internal x264 settings. And my --quick option uses the same strategy. But the x264 encoder has many knobs that you can twist to degrade accuracy and compression efficiency. The trick is avoid those settings which lead to easily perceptible quality loss.

You can see which internal settings are changed for each preset step by looking at the x264 built-in help via:

x264 --fullhelp

Let's look at the first preset, fast. It changes four settings to:

rc-lookahead=30
ref=2
weightp=1
subme=6

The first change, rc-lookahead=30, lowers the number of future frames used for ratecontrol decisions from the default value of 40 to 30. My --quick option makes the same change since it's a modest speedup and doesn't seem to reduce quality. However, lowering rc-lookahead any more will definitely lead to perceptible quality loss. And the other x264 presets continue to step down the rc-lookahead value until it's zero for both superfast and ultraffast.

The next change, ref=2, lowers the number of reference frames available for inter-frame compression from the default value of 3 to 2. This is a significant speedup because it lowers compression efficiency. My --quick option takes this a step further and lowers this to a single reference frame, which the veryfast, superfast and ultraffast presets also do. Using a single reference frame is a strategy also used by hardware-based H.264 encoders.

(BTW, often when people set ref=1 like I do, they also set mixed-refs=0, explicitly disabling per-partition reference decisions. But this is unnecessary since x264 will disable that setting for you automatically.)

The next change, weightp=1, doesn't make much of a performance improvement, nor does it impact quality all that much.

But it's the last change, subme=6, that starts the x264 presets down the road of quality problems. This setting controls the subpixel motion estimation and mode decision behavior. The default value is 7 and should never, ever be lower than 6 if you want to maintain quality.

Even with subme=6 you'll begin to see blockiness and artifacts, usually in indistinct areas of your video first. But lower subme is a significant speedup, which is why all the x264 presets touch it until it's zero for ultrafast.

But my --quick option doesn't change subme. Instead, it gets most of its speedup by disabling partition analysis with partitions=none. Oddly enough, the only x264 preset that pulls this same stunt is ultrafast.

What are partitions? Basically it's a way to divide up a frame so the encoder can make different ratecontrol decisions within that frame. Disabling partition analysis means that that a single decision is made for the entire frame. Which is obviously much faster. And you can get away with it most of the time, especially at higher bitrates.

Here are the three x264 settings I change with the --quick option in order of their performance impact (most to least) and risk of affecting quality (also most to least):

partitions=none
ref=1
rc-lookahead=30

And this simple configuration is actually faster than the faster preset.

The --quick option looks fine for most content at the default bitrate targets. But it's much safer to use with a higher bitrate. In fact, I'm using --abr --target big --quick --prefer-ac3 to re-encode my entire library now.

And I recommend that you also use --target big when you use --quick if you want to play it safe.

I hope that helps. Let me know if you have any other questions.

steveowashere commented 6 years ago

Thanks this is exactly the sort of thing I was looking to read about. Now I know exactly what --quick is doing, I'll keep using it but with more confidence!

You mentioned using --abr --target big --quick and you use it yourself, so it's probably good enough for my purposes, but I've also been using the --target 1080p=12000, just curious if that brings anything to the table. You mention that it's safer to have a higher bitrate and generally 12000 kbps is my 'go to' bitrate for 1080p. I've done a few test encodes are different target bitrates and taken screenshots and used image difference to compare what is being lost vs. the Blu-Ray Source and 12000 is about right for my taste. I don't know what kind of wrench that --target throws into the works.

lisamelton commented 6 years ago

@steveowashere There's no need to increase your target bitrate that much! In fact, it's dangerous for playback compatibility on many streaming devices! Plus, if you're using my default ratecontrol system instead of --abr, your peak video bitrate can really soar if you set the target that high.

Roku and other popular streaming devices have a 10 Mbps (or 10000 Kbps) limit on average video bitrate with a peak video bitrate limit 1.5 times that average, i.e. 15 Mbps (or 15000 Kbps).

One of the reasons I worked so hard on the new --abr implementation is that I wanted to guarantee that average and peak output video bitrates would fit within these typical streaming device limitations. And they always do when you use --target big. Even --target 1080p=10000 will always honor the peak video bitrate limits although the average video bitrate may creep slightly above 10000 Kbps.

But I can't guarantee that with the default ratecontrol system since its goal is to achieve quality at all costs. Even with the default bitrate targets (i.e. 6000 Kbps for 1080p content) it's possible---albeit unlikely---for average and peak video bitrates to exceed the typical streaming limits. And it's obviously an even greater risk using --target big with its setting of 8000 Kbps for 1080p.

This situation definitely happens when transcoding certain episodes of "Battlestar Galactica" (the 2004-era TV series), particularly episode 10 of season 3.

So, using --target 1080p=12000 is a really bad idea. :)

And it's really unnecessary. You're getting sufficient quality with --target big. I suppose you could use --target 1080p=10000 if you're not convinced, but only do that with --abr. It's not safe with the default ratecontrol system.

Now you know why I've never added a --target huge or something like that even though it's been requested many times. :)

steveowashere commented 6 years ago

So, using --target 1080p=12000 is a really bad idea. :)

Hahaha. So glad I asked then! Of course in my mind more = guaranteed quality, when In fact it's not really the case. Thanks once again for taking the time to explain the logic. Learned a lot from this thread. I won't be using --target 1080p=12000 anymore. 👍

That's all the questions I have in mind, so you can close this issue :)

lisamelton commented 6 years ago

@steveowashere Well, a higher bitrate almost always means higher quality. But there are times when it's just not appropriate to raise the bitrate so much. And really high bitrates go against the point of transcoding to make your video smaller and more portable. :)

Thanks for the interesting questions! I enjoyed our conversation so feel free to open other issues if you think of something else to ask.