x264 settings are bad

[kierank]

Please contact x264 developers to discuss further.

The results are pretty meaningless when settings are chosen without an understanding of what they do.

[alvestrand]

I kind of expected this, because it always arises when discussing codecs. The settings were taken from previous suggestions by x264 proponents.

Please add the contact you wish to be contacted for consultation.

[PaulWilkins]

I agree kierank, all settings need to be validated and subject to adjustment and re-run. There are also problems with the VP9 settings though.

For VC / real time any use of frame should not be allowed. Threading is fine provided it is within frame and not across multiple frames. Also in all the tests for a given condition, command line parameters need to be consistent for all the clips and all points in a graph. This is not the case a the moment.

I notice some cases where two points in the same graph have different command lines and this needs to be addressed.

As things stand I think this can only be seen as a place holder for future work and comment and don't think that the results (as of yet) are not that meaningful.

[alvestrand]

The current numbers and graphs are pointed at "what are the best numbers that can be achieved by this codec for these test conditions". It makes sense to do other graphs that go after "what are the numbers that result from holding a certain configuration largely constant across multiple tests", but that's not what the current graphs are looking for.

[PaulWilkins]

I certainly think it makes sense to say if a codec could not achieve the test criteria (i.e for a given point it was too slow or it did not comply with the required buffering constraints) but otherwise think point to point tweaks of the basic parameters are problematic.

For example changes to the real time cpu-speed setting in VP9 will only be valid for the particular machine you ran on on a particular day. A few days later with a new optimization patch or run on a slightly faster or slower machine and the results will change RADICALLY. Also, I noticed a couple of cases where more basic differences such as one point vbr and the other cbr were evident.

In the 264 case for example I noticed some real time points using lookahead 30 and others in the same graph lookahead 60. I am not sure the exact impact of this, other than look-ahead not being sensible for live / VC (though fine for a near live stream).

I would prefer to see an agreed command line for a test case for each codec and then the best case results (with failures points highlighted if need be for real time... but in that case the detailed hardware specification for the machine used to test needs to be given).

I any real world case (like YouTube) you have to come up with a set of settings to use. You cant change them on a point by point and clip by clip basis.

If we have a command line for a particular use case interested parties can comment or suggest improvements, and then hopefully reach consensus. If every point on every clip is hand tuned I think this will be very hard to achieve.

I also think we should extend the test set and perhaps have a different test set for the real time VC case. Just my $0.02 wirth.

[alvestrand]

I have now implemented (not yet committed) an "information" sub-page that will give you one graph for each configuration used in the overall comparision, and some visualization help (choice to graph by score or CPU time used rather than by PSNR) that allows the viewer to understand why a particular configuration was considered "best" at a given rate target. This might help.

[alvestrand]

I have now added a single-configuration graph set as the first result in the lists - PR #26. Will close this bug once that PR lands.

[alvestrand]

PR landed. Closing bug.