Proposal to increase the speed and quality of the `--quick` option

[lisamelton]

Note: This comment has been edited multiple times since it was first posted due to changes in the proposed implementation. You can track its evolution via the comments below.

Proposal to increase the speed and quality of the --quick option (revised)

The --quick option has been part of transcode-video since the beginning. It has allowed users to trade some precision, i.e. perceptible video quality, for a 45-50% increase in encoding speed. Most of the time, this was a good bargain. And any quality loss using --quick was still less than what was perceptible when using the faster and veryfast presets of the x264 encoder.

However, --quick was still slower than --preset faster and there are cases where quality loss was quite noticeable. For example, the second chapter of "Jaws (1975)" and the sixth chapter of "Gone Girl (2014)," both on Blu-ray Disc, exhibit blockiness and artifacts in their backgrounds when using --quick.

So, I have now completely rewritten the implementation of --quick to address these performance and quality issues.

My proposed change will increase encoding speed 70-80% over the default with no easily perceptible loss in video quality. Seriously. This means --quick will now be much speedier than --preset faster and, in some cases, you might even like the appearance of --quick better than the default.

Please try this proposed implementation yourself. Don't use --quick itself since that will invoke the current implementation. Instead add four new --encoder-option (or -E) arguments to your command line like this:

transcode-video -E partitions=none -E ref=1 -E mixed-refs=0 -E rc-lookahead=30 "/path/to/Movie.mkv"

And let me know what you think. Thanks.

[CameronBanga]

I don't have chance to try tonight, and am not a heavy user, but I want to give emphatic yes on this. I don't see a big negative here, and speed boost is always appreciated. I can't see much of a downside.

[platonium]

Thanks, Don, this sounds great. Why not make the new —quick the default for transcode-video itself?

[lisamelton]

@platonium As always, my plan is to take over the world. :) I think it's best to start this behavior, because it's so radical, as an option before making it the default. But if everyone prefers it once they see it then, yes, we can make it the default.

[lisamelton]

@platonium BTW, a better new default behavior might be just the first half of those options, i.e. -E mbtree=0 -E partitions=none. That's still faster than the old --quick option but it retains the three reference frames of the current default.

[edrozenberg]

I haven't tried it yet but it sounds good :). As for defaults, keeping the quality as high as possible sounds like the right approach - since encodes take so long anyway, nobody's really counting the hours at that point and the important thing is to have the best possible output without having to re-encode again in the future.

[lisamelton]

@edrozenberg Yes, I would rather stick with the quality, i.e. non-ratecontrol, defaults of x264, which is the current behavior. But please try these out and let me know what you think.

[lisamelton]

Folks, please note the update to my original proposal.

[edrozenberg]

Rather than calling the options by their speed (and letting people guess as to quality), what about naming them by their quality and having a note in the docs that better quality means X% more time to encode. Normal, decent, and passable are not the best terms (since all 3 options are pretty good) but as a discussion starter:

• Normal quality: no CLI flag, or optionally --q_good or --q_normal

• A bit less good: --q_decent: transcode-video -E mbtree=0 -E partitions=none "/path/to/Movie.mkv"

• Two bits less good: --q_passable: transcode-video -E mbtree=0 -E partitions=none -E ref=1 -E mixed-refs=0 "/path/to/Movie.mkv"

[lisamelton]

@edrozenberg Interesting idea. But it just doesn't seen very enticing to the user to ever try the option if they don't understand it's a performance improvement. And in this case, with very little downside for quality.

[CameronBanga]

+1 for Turbo mode. :)

[bmhayward]

@donmelton doing some quick tests with the settings, -E mbtree=0 -E partitions=none -E ref=1 -E mixed-refs=0, seeing anywhere from 58-73 percent performance increase when compressing. File sizes go up a little and quality, for my eyes stays almost the same. There is occasionally a little more "grain" to the video with the updated settings.

I'll test these settings in a bit, -E mbtree=0 -E partitions=none, compare and let you know.

All-in-all, looks really good!

[lisamelton]

@bmhayward Thanks for testing!

Yes, you should see a little more grain. That's actually a feature. :) Disabling macroblock ratecontrol with mbtree=0 and macroblock partition analysis with partitions=none prevents x264 from creating as many "dancing pixels" which allows you to perceive the grain better.

[lisamelton]

@bmhayward BTW, try encoding movies like "Young Frankenstein (1974)" with coarse grain or "300 (2007)" with fine grain to see which versions you like better.

[lisamelton]

@bmhayward On your performance numbers: how many CPU cores do you have?

[bmhayward]

@donmelton I'm testing on a iMac with a 4 core 3.4GHz i7.

[lisamelton]

@bmhayward Ah. That's probably it. I have more cores then. HandBrakes reports 8 logical CPUs for me.

[bmhayward]

@donmelton testing with the -E mbtree=0 -E partitions=none, yields a 28 percent longer compress time and a 0.36 percent larger file when compared against the -E mbtree=0 -E partitions=none -E ref=1 -E mixed-refs=0.

However, the video quality to my eye is the same as the original with these settings.

Comparing against the defaults for video-transcode, the compress time with these settings is 68 percent faster for this particular video and yields a 1.08 percent increase in file size.

I would say, this is the best compromise for file size, compress time and quality.

[bmhayward]

@donmelton Handbrake also reports 8 logical CPU's

[lisamelton]

@bmhayward OK, there must be something else odd going on then. I've done my measurements with about 25 full-length movies, transcoded both ways, while my Mac wasn't doing anything else.

[bmhayward]

@donmelton it's all good. I'll take any form of 50 percent or greater increase in compression speed with no detectable change in quality!

[lisamelton]

@bmhayward Haha! Yeah, that's what I thought when I figured this out. :)

[lisamelton]

After another day of testing, I have found a case where some quality loss might be perceptible.

It's for a few frames during a 12-second cut in the non-Criterion version of the "Moonrise Kingdom (2012)" Blu-ray, toward the end of chapter six. This is a scene with very blocky grain, meaning the original encoding is dodgy, which is probably why Criterion re-issued the disc. And, to be fair, this scene doesn't look perfect when transcoded with my defaults either.

Overall, the "Moonrise Kingdom (2012)" Blu-ray actually might look slightly better when transcoded with my proposed changes to --quick than using the defaults because finer grain is retained without as many "dancing pixels." But it's a close call.

Please let me know if you find any questionable scenes in other discs. With such a large collection, there's a good chance I have the same content so I might be able to reproduce it.

Thanks.

[bmhayward]

@donmelton how are you testing? I would like to follow the same test methodology if possible.

[lisamelton]

@bmhayward I do my testing of transcoding in two environments or phases.

During the first phase, usually during the day, I have other applications open so my system is not in a quiescent state. This means any performance data I get has to be revalidated later. But evaluating quality and other metrics are fine. I'm also usually transcoding single problematic chapters from certain movies instead of whole titles. This allows me to do A/B comparison testing reasonably fast.

During the second phase, usually at night, I quit all other application except Terminal.app so my system is in a quiescent state before I start batch transcoding whole titles. This means performance data is valid along with the other metrics.

To evaluate performance, I compare FPS data using query-handbrake-log speed. It's much easier than comparing total time.

[lisamelton]

Aaaaand... after another day and night of testing, I've found a way to eliminate the perceptible quality loss problem I found with the non-Criterion version of the "Moonrise Kingdom (2012)" Blu-ray.

If I don't disable macroblock ratecontrol with mbtree=0 then the issue goes away completely. In other words, if I change the command line to:

transcode-video -E partitions=none -E ref=1 -E mixed-refs=0 "/path/to/Movie.mkv"

The bad news is that it's slightly slower. The bonus good news is that the output is smaller. In fact, it's smaller than using the default settings.

Of course, these speed and output size differences apply to all videos when using the new command line.

But what does it mean for quality in other videos? Generally, finer grain is still retained but there may be more cases of "dancing pixels" here and there. In a way, the output looks much more like using the default settings.

So, I'm going to test this for a few more days and report back here. Stay tuned.

[lisamelton]

What a stupid I am! I can regain most of the performance lost from disabling macroblock ratecontrol by reducing the default ratecontrol lookahead from 40 frames to 30 frames via rc-lookahead=30. In other words, if I change the command line to:

transcode-video -E partitions=none -E ref=1 -E mixed-refs=0 -E rc-lookahead=30 "/path/to/Movie.mkv"

... it's almost as fast as the original proposal. And the output size is about the same as using the default settings.

I didn't mess with the lookahead setting back when I was also disabling macroblock ratecontrol because the two are tied together and such a small change wouldn't affect performance very much. Besides, you don't want to set it lower than 30 frames because that would reduce the stability and accuracy of the VBV.

Technically, rc-lookahead is a quality setting but after transcoding numerous samples, I'm not seeing any perceptible degradation in quality. So, I think it's safe.

Again, I'm going to test this for a few more days and report back here. Stay tuned.

[lisamelton]

OK, I now have 20 Blu-ray Discs movie rips transcoded with partitions=none:ref=1:mixed-refs=0:rc-lookahead=30 and everything looks good so far. It's very difficult, if not impossible, for me to tell them apart from transcodings using the default settings, even viewing them a foot away from my 27-inch Retina 5K iMac screen.

Some transcoded even faster than using the original mbtree=0-based proposal. But they're all within a minute or two of each other. And, so far, the performance gain is around 70-80% above the default settings.

Output size/bitrate is usually a few kilobytes below the default settings and occasionally even fewer above. In other words, so close that it doesn't matter. But definitely smaller than the original mbtree=0-based proposal.

I'll keep at this awhile longer, but I think I've found the winning formula for the --quick option now:

transcode-video -E partitions=none -E ref=1 -E mixed-refs=0 -E rc-lookahead=30 "/path/to/Movie.mkv"

I plan to describe it this way in the built-in help:

    --quick         increase encoding speed by 70-80%
                      with no easily perceptible loss in video quality
                      (avoids quality problems associated with x264 presets)

Let me know what you think, especially about the "no easily perceptible loss in video quality" wording.

I'll probably update the original proposal comment at the top of this thread later today or tomorrow.

Thanks for your patience!

[lisamelton]

Folks, please note that I've now revised my original proposal comment at the top of this thread.

[platonium]

Thank you again for the hard work. I rarely transcode very large batches or re-transcode so I’d probably lean towards quality in most cases.

That being said, if you can hardly perceive the difference when actively trying. I highly doubt I’d spot it so I’ll definitely keep the new setting in mind if I ever run into this kind of time requirement.

[lisamelton]

@platonium It is, indeed, handy when you're in a hurry. :) And you are very welcome, sir.

[SFoskett]

The output looks very good to me. And the performance increase is notable once it really gets moving. For some reason it was still a little slower toward the beginning of the encode...

[lisamelton]

@SFoskett Good to hear although I'm not sure why it would be slower at the beginning. That's a little weird. I'll take a look at my logs and see if I have any behavior like that as well.

[lisamelton]

I now have 15 more Blu-ray Discs movie rips transcoded with partitions=none:ref=1:mixed-refs=0:rc-lookahead=30 and these latest 15 look just as good as the first 20. No problems so far. Steady as she goes, Mr. Sulu.

[lisamelton]

OK, the code has been checked in and released as version 0.12.3.

Thanks again for all the feedback!