Custom interface for rendering subtitles on an RGBA texture(s) | (XySubFilter for madVR) [Part 2]

[GoogleCodeExporter]

Original Issue: http://code.google.com/p/xy-vsfilter/issues/detail?id=40

Original issue reported on code.google.com by cyber.sp...@gmail.com on 30 Apr 2012 at 10:22

• Merged into: #152

[GoogleCodeExporter]

> If the user knew their video was BT.800, they would fix the script as follows 
to be correct:

> sourceMatrix: BT.800
> yuvMatrix: BT.709

This is made under the assumption that the user knows about matrices and stuff, 
in which case it seems more appropriate to have him:
- correctly write matrix to video bitstream or
- have him select the correct yuvMatrix instead of using any workarounds

Original comment by sneaker...@googlemail.com on 4 May 2012 at 8:47

[GoogleCodeExporter]

> If we use explicit matrix tagging then with VSFilter YCbCr rendering the
> subtitle colors would match the video colors regardless of whether the 
> user selects the correct matrix in madVR or not.

That's how it should be.

This is a problem for madVR to come up with a solution for.

> We need to make sure YCbCr and RGB rendering always produce identical
> results. It makes no sense to have them differ.

Always is a strong word. I don't see a problem with YCbCr rarely being more 
correct than RGB, when the only other option is them both being wrong.

> *If* we decide that RGB is the law and YCbCr rendering should mimick it,
> then VSFilter and xy-vsfilter need to use the inverse of the video 
> renderer's matrix to convert the ASS RGB colors to YCbCr, because 
> that's the only way to make sure that the YCbCr rendering method 
> will produce the same results as untouched RGB rendering in all situations.

But yes, the goal would be for xy-vsfilter to use the inverse of the video 
renderer's matrix to convert the ASS RGB colors to YCbCr.

It will only produce the same results as untouched RGB if madVR uses the 
correct matrix. The problem is how do you get madVR to use the correct matrix 
when it wouldn't otherwise, without using explicit tagging?

Original comment by cyber.sp...@gmail.com on 4 May 2012 at 8:49

[GoogleCodeExporter]

[deleted comment]

[GoogleCodeExporter]

> This is made under the assumption that the user knows about matrices and 
stuff,
> in which case it seems more appropriate to have him:
> - correctly write matrix to video bitstream or
> - have him select the correct yuvMatrix instead of using any workarounds

Of course it would.

If you forget to specify matrix during encoding, writing the matrix to a video 
bitstream is an advanced task, which may or may not be easily possible to do 
depending on the format. Mistakes happen.

Setting the correct yuvMatrix would be more feasible, but only if Aegisub 
allowed overriding the matrix to "BT.800" like it does with "BT.601".

That's why I've recently become fond of:

sourceMatrix: TV|PC.Matrix <--- Optional
yuvMatrix: TV|PC.Matrix <--- Required
rgbCorrection: True|False <--- Required

Original comment by cyber.sp...@gmail.com on 4 May 2012 at 8:59

[GoogleCodeExporter]

> If you forget to specify matrix during encoding, writing the matrix to a 
video bitstream is an advanced task, which may or may not be easily possible to 
do depending on the format. Mistakes happen.

Sure, but that's too far stretched, IMHO. I don't think we should create extra 
possibilities for people creating wrong files. If people want to have correct 
colors on their non-SD-BT.601/non-HD.BT.709 videos (even without any subtitles) 
they will always have to mark them correctly in the video bitstream. That's 
beyond the scope of the subtitles.

Original comment by sneaker...@googlemail.com on 4 May 2012 at 9:10

[GoogleCodeExporter]

[deleted comment]

[GoogleCodeExporter]

> You seem to be saying you rather both YCbCr output
> and RGB output match each other but not the video,
> than YCbCr alone being correct because correct RGB
> output is impractical.

That is not my point. My point is that a spec should be clear. And all 
implementations should follow the spec. As a result all implementations 
(regardless of internal logic differences) must produce the same results.

Come on, this is the very base of what makes a spec a spec!

> That equals leaving VSFilter broken.

Sorry, but that's just not true. You seem to think that the color matrix used 
by the subtitle file must match the one used by the video file. But that is not 
true at all. If Aegisub always produces BT.601 ASS files and VSFilter always 
renders them as BT.601 then the results will always be perfect. From a 
practical point of view there's nothing broken about it at all. It might not be 
a "pretty" solution, but the end user would always get perfect results.

-------

I see 3 possibilities about what the spec for "new" ASS files could be:

(1) We judge ASS colors to be native RGB.
(2) We judge ASS colors to be native YCbCr, always hard coded as BT.601.
(3) We judge ASS colors to be native YCbCr, using a matrix which matches the 
video.

Let's look at the advantages and disadvantages of every of these options:

(1) We judge ASS colors to be native RGB.

+ very easy to understand
+ the most "logical" solution, because ASS colors are stored in RGB format
+ identical to libass interpretation
+ extremely easy to implement for new RGB interface
+ same subtitle file will work for dual HD/SD releases
- vsfilter YCbCr rendering would be more difficult to do right
- subtitle colors will only match the video if creation and playback matrixes 
match
- special treatment needed for "old" ASS files

(2) We judge ASS colors to be native YCbCr, always hard coded as BT.601.

+ subtitle colors will always match video, no matter what
+ compatible to "old" ASS files, so no special treatment needed
+ extremely easy to implement for vsfilter
+ same subtitle file will work for dual HD/SD releases
o average implementation complexity for new RGB interface
- a bit weird, cause ASS colors are stored in RGB
- not compatible with current libass interpretation for non BT.601 videos
(- subtitle matrix can differ from video matrix *)

(3) We judge ASS colors to be native YCbCr, using a matrix which matches the 
video.

+ subtitle colors will always match video, no matter what
(+ subtitle matrix will usually, but not always, match video matrix *)
o average implementation complexity for vsfilter (needs to know and support 
different matrixes)
o usually, but not always, compatible with current libass interpretation
- same subtitle file will *not* work for dual HD/SD releases
- extra work needed to make sure new RGB interface produces the correct output
- a bit weird, cause ASS colors are stored in RGB

*) This is purely a cosmetical argument and has no practical consequence 
whatsoever.

-------

Whatever we decide, I believe all renderer implementations should be made to 
perfectly follow the spec. Which also means that if explicit matrix tagging 
were to be used, an RGB based rendering implementation would have to apply all 
necessary matrix corrections. Personally, I think explicit matrix tagging is 
the worst option of all 3, though.

Original comment by mad...@gmail.com on 4 May 2012 at 9:17

[GoogleCodeExporter]

My currently favored approach:

color workaround: on|off

1.) case off:
colors are "true" RGB
 a.) for the new interface, vsfilter just renders directly in RGB
 b.) for old decoder->vsfilter->renderer, vsfilter gets video matrix
       - from decoder
       - guesses based on resolution (SD == BT.601, HD = BT.709)

2.) case on:
colors have an assumed BT.601->BT.709 workaround
 a) for the new interface vsfilter corrects colors using the BT.601->BT.709 workaround before sending the RGB to madVR
 b) for the old decoder->vsfilter->renderer way, it corrects colors if the video is not BT.709. Actual video matrix is determined as shown in 1.)

Aegisub will start writing this tag immediately. For the near future it will 
always have "color workaround: on" for BT.709 files and "color workaround: off" 
for all other files. When new xy-vsfilter gets accepted into CCCP and has 
reached sufficient number of systems, Aegisub will always use "color 
workaround: off" by default. (The option to turn it on or off will remain for 
some extra time, though)
Also Aegisub should make sure it can handle both TV and PC input (both YUV PC 
and RGB TV have to converted to RGB PC for working in the Aegisub preview). 
Having Aegisub apply the color workaround to the subtitles instead of the video 
colors to make it fully WYSIWYG would be nice, if someone finds the time to 
implement it. A small tool to convert between workaround on and off would also 
be great.

If the tag is non-existant, xy-vsfilter will assume "Color Workaround: on", 
though allow users to turn that off in the options.

Pros:
- very simply to understand and implement
- mid to long term compatible with VLC, mplayer2 etc.
- script's colors are unambiously defined by itself and completely separate 
from video info. All tracks remain independent. Results in:
    * scripts can be remuxed without changes
    * scripts can be rendered by itself, implementations that use RGB rendering (VLC, mplayer2, MPC-HC and future new xy-vsfilter interface) don't need to learn new matrices

Neutral:
- new workarounds are not possible. (I put this in "neutral" because I don't 
want to encourage anyone to use new workarounds, and I don't think new 
workarounds are likely to appear because of the several RGB implementations 
that are available now)

Cons:
- If anyone actually used a BT.601->non-BT.709 workaround, his scripts are 
broken. (Never actually saw anyone using it though, probably affects 0.0x% of 
files. negligible IMHO.)
- if Aegisub or xy-vsfilter guess a matrix wrong or don't know it, it can 
result in wrong colors *

* This Con will however apply to other solutions in one way or another as well. 
(Sometimes affecting RGB rendering and sometimes YUV rendering)

Original comment by sneaker...@googlemail.com on 4 May 2012 at 9:40

[GoogleCodeExporter]

> And all implementations should follow the spec.
> Come on, this is the very base of what makes a spec a spec!

Ever since VSFilter did permanent damage to the ASS spec, having all 
implementations follow the spec became impossible. From what I remember, the 
spec isn't even that well defined. At this point I only want to make the spec 
slightly more friendly into the future.

That's why we've been saying for years that we need to create a new spec (like 
AS5), which hasn't been poisoned by VSFilter...

> If Aegisub always produces BT.601 ASS files and VSFilter always renders
> them as BT.601 then the results will always be perfect.

And almost always be wrong in Libass, which makes those users unhappy. 

> (3) We judge ASS colors to be native YCbCr, using a matrix which matches the 
video.

I still favor (3). The only problem which I actually consider a problem out of 
your disadvantages, is the potential for extra work needed for the new 
interface.

What did you think of the following?

sourceMatrix: TV|PC.Matrix <--- Optional
yuvMatrix: TV|PC.Matrix <--- Required
rgbCorrection: True|False <--- Required

In case the above tagging solution is not clear.

"rgbCorrection: False" means yuvMatrix = sourceMatrix (not tagged)

"rgbCorrection: True" means yuvMatrix = matrix used in Aegisub

Only with "rgbCorrection: True", would tagging sourceMatrix be optional. 
Otherwise, the fallback is reading the bitstream matrix or guessing.

At this point in time, the extra work created by "rgbCorrection: True" 
shouldn't be too bad if we just limit ourselves to current possibilities within 
Aegisub. In other words, we could restrict "yuvMatrix" to BT.601 when 
"rgbCorrection = True" until further notice. With "rgbCorrection = False", we 
could only support cases where "yuvMatrix" matches the bitstream or auto-guess 
choice.

In the future we could gradually expand what we support, without needing to 
re-define the ASS spec again.

Original comment by cyber.sp...@gmail.com on 4 May 2012 at 10:21

[GoogleCodeExporter]

[deleted comment]

[GoogleCodeExporter]

small corrections for #58:
"2.) [...]
 b) for the old decoder->vsfilter->renderer way, it corrects colors if the video is not BT.709. Actual video matrix is determined as shown in 1.)"
ought to be
"2.) [...]
 b) for the old decoder->vsfilter->renderer way, it corrects colors if the video is not BT.709. (Actual video matrix is determined as shown in 1.)) Script is rendered to YUV using BT.601 in the case of BT.709 video."

"If the tag is non-existant, xy-vsfilter will assume "Color Workaround: on", 
though allow users to turn that off in the options."
ought to be
"If the tag is non-existant and video colorspace == BT.709, xy-vsfilter will 
assume "Color Workaround: on", though allow users to turn that off in the 
options."

Original comment by sneaker...@googlemail.com on 4 May 2012 at 10:45

[GoogleCodeExporter]

@sneaker, I could live with that quite well.

@cyber,

> Ever since VSFilter did permanent damage to the
> ASS spec, having all implementations follow the
> spec became impossible.

But if try to improve the spec by changing it, we should aim to make things 
better. Meaning we should make it clear how colors are to be interpreted, and 
we should demand that all renderers produce the same output. Otherwise I 
believe changing the spec will only make things even worse than they already 
are.

> And almost always be wrong in Libass, which
> makes those users unhappy. 

They could still complain to the Libass devs and hope for a fix.

> I still favor (3). The only problem which I
> actually consider a problem out of your
> disadvantages, is the potential for extra
> work needed for the new interface.

That is a certain problem. Maybe not even for me, but for every potential 
developer who might consider using the new RGB interface. Such complications 
could scare him away.

> What did you think of the following?
> sourceMatrix: TV|PC.Matrix <--- Optional
> yuvMatrix: TV|PC.Matrix <--- Required
> rgbCorrection: True|False <--- Required
> "rgbCorrection: False" means yuvMatrix = sourceMatrix (not tagged)
> "rgbCorrection: True" means yuvMatrix = matrix used in Aegisub

And here comes the problem: Even after all these discussions, I'm not sure what 
the exact consequences of these fields being set how would be. If sourceMatrix 
is different from yuvMatrix, what does vsfilter have to do for YCbCr rendering? 
What does the new RGB interface have to do? <- these are rhethorical question. 
Just trying to say that I find this awfully complicated and hard to understand.

If we do use explicit matrix tagging (which is still my least favorite option), 
I would vote for only storing the Aegisub matrix and nothing more. Why would we 
need any more than that? The Aegisub matrix alone would allow vsfilter to 
render the subtitles in such a way that subtitle and video colors will always 
match. That should be good enough. I don't see what more benefit adding a 
second matrix in there would bring.

Also, I still think we should leave PC vs TV levels out of it. It's already 
complicated enough as it is. Let Aegisub do the necessary conversions if 
something weird is going on with the levels. No need to store that information 
in the ASS file, as far as I can see. It's a corner case, anyway. Pretty much 
all movie content out there is video levels.

Original comment by mad...@gmail.com on 4 May 2012 at 11:17

[GoogleCodeExporter]

[deleted comment]

[GoogleCodeExporter]

[deleted comment]

[GoogleCodeExporter]

@sneakerger

Remember that I want to have the "yuvMatrix" tagged to simplify YCbCr output. ;)

Otherwise, the behavior you explain seems mostly identical to what I'm 
proposing to start with as well in Comment #59. The main difference being what 
you list under "Neutral:" for expanding functionality in the future, as well as 
additional potential for ensuring that Subtitle Editor + Subtitle Filter + 
Video Renderer are all using the correct matrix.

How much of that potential functionality we ultimately decide to implement will 
depend on what the future holds, as well as user desires. I see it more as 
having a safety net for worse case scenario, rather than encouraging 
workarounds.

@madshi

> But if try to improve the spec by changing it, we should 
> aim to make things better. 

I understand what you're saying, but xy-VSFilter's goals for that are limited 
to positive side-effects of fixing stupidity in VSFilter. The ASS spec is way 
more broken than just this BT.601 matrix problem. This matrix problem is just 
one of the rare things we can actually fix, without breaking legacy VSFilter 
compatibility.

> They could still complain to the Libass devs and hope for a fix.

They claim what they are doing is "correct", and what VSFilter does is "wrong".

In other words, they see no need to fix anything.

> If sourceMatrix is different from yuvMatrix, what does vsfilter have 
> to do for YCbCr rendering?

yuvMatrix would always be used for YCbCr output.

sourceMatrix we define as completely optional, and a value never tagged by 
Aegisub. Basically just something to have available if needed in the future 
where it could potentially be useful for RGB output to eliminate matrix 
guessing. Not a problem today, so we may never need to support it at all.

> What does the new RGB interface have to do? 

In that case, VSFilter would output double-corrected RGB, and the video 
renderer matrix when alphablending the subtitles would need to match 
sourceMatrix. Again, we don't need to support sourceMatrix at all currently, 
since it's not really needed today.

> I would vote for only storing the Aegisub matrix and nothing more.

That's what I'm proposing. And as stated in Comment #59, it would function 
nearly identically to snearkger's idea which you like.

> Also, I still think we should leave PC vs TV levels out of it. It's already 
complicated enough as it is.
> Let Aegisub do the necessary conversions if something weird is going on with 
the levels. No need to
> store that information in the ASS file, as far as I can see. 

I think you are misunderstanding something, since having levels specified for 
YCbCr output won't affect RGB output, or the RGB values the script at all. 
There are no "necessary conversions" for Aegisub to do which could solve this. 

Currently VSFilter always does PC->TV levels conversion when outputting YCbCr, 
but since script RGB values are PC-range, there is no hurdle to supporting 
PC-range YCbCr output, other than knowing when to do so. 

Original comment by cyber.sp...@gmail.com on 4 May 2012 at 12:21

[GoogleCodeExporter]

madshi, if I'm not seriously mistaken about the different YUV matrices, then 
the HD version (BT.709) should result in the same colours as the SD version 
(BT.601), right? If that is so, then there are some corrections to be made 
(taken from post #57):
> (1) - subtitle colors will only match the video if creation and playback 
matrixes match
I think the creation matrix doesn't have to match the playback matrix, as long 
as both matrices are correct for the corresponding video file. The video file 
used to create the ASS file was decoded to RGB using Matrix A, which is the 
correct matrix for that file. This does not mean that the subtitles need to be 
converted to YUV using Matrix A and be overlaid onto the final version of the 
video if that version's actually using Matrix B, as that would lead to a colour 
distortion (similar to what we're trying to do to create correct output for the 
files using the BT.601 workaround). Instead, xy-VSFilter should use the video's 
matrix, Matrix B, to convert the ASS file to YUV and then overlay it, which 
will create correct colours.

> (2) + same subtitle file will work for dual HD/SD releases
see my comment to (1), because the HD release stores the same colours in a 
different matrix, it will not work for dual HD/SD releases (unless, of course, 
both releases use the same matrix tagged to the bitstream, but I don't think 
that's what you meant).

> (3) - same subtitle file will *not* work for dual HD/SD releases
see (1) and (2), it's the contrary, the files will work.

The yuvMatrix (without SourceMatrix) approach suffers from the same problem, if 
the SD video uses another matrix than the video used to create the ASS file, 
the colours will be wrong. Instead, xy-VSFilter should get the matrix used from 
the renderer (or the bitstream) and use that.
If we can conclude that "rgbCorrection: on" always has to mean RGB --> YUV 
(using TV|BT.601) --> RGB (using TV|BT.709), we can even completely remove 
every mention of yuvMatrix and SourceMatrix, because any value other than 
SourceMatrix: TV|BT.601 and yuvMatrix: TV|BT.709 (implied by "rgbCorrection: 
on") or "rgbCorrection: off" would be invalid, with "rgbCorrection: off" 
meaning to use the video's matrix for YUV output (if it is necessary, you could 
even implement a "rgbCorrection: on,PC" switch for those special cases where TV 
range BT.601 would be layered upon a PC range BT.709 video).
Also, I'm in favour of sending the (preferably high-bitdepth, as I understand 
it, RGB --> YUV conversions aren't lossless) BT.601 YUV data to madVR as is in 
case of a legacy VSFilter file and have madVR convert the YUV to RGB using 
BT.709 before overlaying it onto the video in order to avoid possible rounding 
errors.

Original comment by TheDarkS...@gmail.com on 4 May 2012 at 1:24

[GoogleCodeExporter]

> That is a certain problem. Maybe not even for me, but for every potential
> developer who might consider using the new RGB interface. Such complications
> could scare him away.

Overall, this method still seems very logical to me. Today, all cases which 
YCbCr output uses the correct matrix should also be Untouched RGB.

Under that assumption, I think it's unrealistic to expect other subtitle filter 
developers to support "Corrected RGB" at all, which is why we should be moving 
towards using "Untouched RGB" the majority of the time.

The only times we shouldn't use "Untouched RGB" ourselves, would be legacy 
compatibility, and potentially in the future situations where "Untouched RGB" 
matching the video becomes impossible. 

It shouldn't be difficult for Aegisub + xy-VSFilter + madVR to work together 
defining supported scenarios in sync with each other, in order to limit 
complexity. The only change I could see being made to the new interface 
specification, would be a provider set matrix value. At some point we will need 
to decide if we want the functionality that would provide or not.

Original comment by cyber.sp...@gmail.com on 4 May 2012 at 1:24

[GoogleCodeExporter]

> They claim what they are doing is "correct",
> and what VSFilter does is "wrong".
> In other words, they see no need to fix anything.

Sure. But if enough users complain, they'll eventually get heard. The libass 
devs can't deny that 99.9% of the ASS files out there where created with an 
Aegisub/VSFilter version which was hard coded to BT.601. If enough users 
complain I think they might at least consider adding an option to behave like 
VSFilter.

> That's what I'm proposing. And as stated in
> Comment #59, it would function nearly identically
> to snearkger's idea which you like.

No. AFAIU Sneaker's preferred solution is to treat ASS subtitles as native RGB 
subtitles, while you want to treat them as native YCbCr subtitles. There's a 
world of a difference between those two things.

> I think you are misunderstanding something, since
> having levels specified for YCbCr output won't
> affect RGB output

It's not as easy as you say. I'm already getting headaches again. What happens 
if the ASS file says the subtitles are full range, but the video renderer 
thinks the video file is limited range? That would result in YCbCr and RGB 
render paths to produce different results, would it not?

Original comment by mad...@gmail.com on 4 May 2012 at 1:30

[GoogleCodeExporter]

> which is why we should be moving towards using
> "Untouched RGB" the majority of the time

Then why not be 100% consequent about it and use option (1) from my post #57? I 
would prefer that. sneaker seems to prefer it, too. And from my memory, 
nevcairiel and tgoyne also preferred that over explicit matrix tagging.

Original comment by mad...@gmail.com on 4 May 2012 at 1:34

[GoogleCodeExporter]

> No. AFAIU Sneaker's preferred solution is to treat ASS subtitles 
> as native RGB subtitles, while you want to treat them as native 
> YCbCr subtitles. There's a world of a difference between those two things.

> Then why not be 100% consequent about it and use option (1) from
> post #57? I would prefer that. sneaker seems to prefer it, too. And from 
> my memory, nevcairiel and tgoyne also preferred that over explicit 
> matrix tagging

There really isn't a world of difference. It's more like a one in a million 
chance of there being a difference. You could easily make option (3) behave 
like option (1), while retaining the added benefit which yuvMatrix offers for 
YCbCr output.

Since we now seem to be getting to the point where this will be useful, I'll 
post the following which I typed up a couple days ago for my own reference.

__________
As of today, there is near-zero expectation of script portability when doing 
color-critical typesetting.

Without script portability:

With "color repair needed: yes" + Aegisub matrix, YCbCr output will always be 
correct (BT.601 Subtitles on non-BT.601 video is unfavorable). RGB output will 
be correct as long as the video renderer and subtitle filter assume/read the 
same video matrix.

With "color repair needed: no" + Aegisub matrix, YCbCr output will always be 
correct. RGB output has the potential to always be correct.

With "color repair needed: yes", YCbCr output will always be correct (BT.601 
Subtitles on non-BT.601 video is unfavorable). RGB output will be correct as 
long as the video renderer and subtitle filter assume/read the same video 
matrix.

With "color repair needed: no", YCbCr output will be correct only if the 
subtitle filter assumes/reads the correct video matrix. RGB output will be 
correct only if the video renderer assumes/reads the correct matrix.

With only Aegisub matrix explicitly tagged, YCbCr output will always be 
correct. RGB output will be correct as long as the video renderer and subtitle 
filter both assume/read the same video matrix.

With both Aegisub matrix + Video matrix is explicitly tagged, YCbCr & RGB 
output will always be correct.

Script portability to video with a different matrix:

With "color repair needed: yes" + Aegisub matrix, YCbCr and RGB output will be 
wrong.

With "color repair needed: no" + Aegisub matrix, YCbCr output will always be 
correct. RGB output has the potential to always be correct.

With "color repair needed: yes", YCbCr and RGB output will be wrong.

With "color repair needed: no", YCbCr will be correct as long as the subtitle 
filter assumes/reads the correct video matrix. RGB output will be correct only 
if the video renderer assumes/reads the correct matrix.

With only Aegisub matrix explicitly tagged, YCbCr and RGB output will be wrong.

With both Aegisub matrix + Video matrix is explicitly tagged, YCbCr & RGB 
output have the potential to always be correct.
__________

Original comment by cyber.sp...@gmail.com on 4 May 2012 at 1:57

[GoogleCodeExporter]

[deleted comment]

[GoogleCodeExporter]

> What happens if the ASS file says the subtitles are full range,
> but the video renderer thinks the video file is limited range? 
> That would result in YCbCr and RGB render paths to produce different 
> results, would it not?

No, actually it wouldn't. The YCbCr and RGB subtitle render paths results would 
be identical.

ASS subtitles always have colors defined in full range RGB. There is no such 
thing as a limited range ASS subtitle file.

The only problem would be madVR double-expanding a PC-range YUV video.

Original comment by cyber.sp...@gmail.com on 4 May 2012 at 2:14

[GoogleCodeExporter]

> It's more like a one in a million chance of there being a difference.

If that's the case then why not choosing (1)? IMHO it's much easier to 
understand, it's more logical, it's probably what the original ASS developers 
had in mind, it's how libass works, and it appears to be the favorite of most 
people who have participated in this thread.

Original comment by mad...@gmail.com on 4 May 2012 at 3:16

[GoogleCodeExporter]

Wow, gone for a few hours and a whole new book has been written...

> @sneakerger

> Remember that I want to have the "yuvMatrix" tagged to simplify YCbCr output. 
;)

Well, "color workaround: on" has this information too, even if it is only 
implicit, because it means the same as:

sourceMatrix: TV.BT.709
yuvMatrix: TV.BT.601
rgbCorrection: True

> There really isn't a world of difference. It's more like a one in a million 
chance of there being a difference. You could easily make option (3) behave 
like option (1), while retaining the added benefit which yuvMatrix offers for 
YCbCr output.

In the end, the decoder->xy-vsfilter->renderer implementation is only one 
possible way to do things. I don't really want to add additional information, 
because I think
 it favors a single implementation. In a case where this would actually be useful, the RGB implementations would get more complicated. Nothing is gained. Instead:
 - script portability is lost (if source matrix is optional)
 - certain chains don't work. Like video decoder being set to RGB output
 - it mixes inherent video attributes into the subtitle file
 - correct rendering needs info about video even with the new interface, the whole chain would need to pass on that information.
 - users who do not create their files and set up their playback chains correctly will always have problems. Trying to work around that is futile.
 - I don't think we want to "gradually evolve" ASS anymore. This is basically a one time only thing to fix a vsfilter problem, nothing more.
 - the simpler the approach, the more likely it is for e.g. VLC and MPC-HC to implement it. (even if the libass devs object it, they only offer a library, not an end user filter)
 - allowing all kinds of different source and aegisub matrices means a lot of work for implementations.
 - what was supposed to be a hint becomes the source of an error when remuxing

I will take the time to read your posts again and be back tomorrow, hopefully 
not stumbling on walls of new text.

But I really don't see the need for a yuvMatrix tag (in addition to the implied 
"on") and am more comfortable with using the information from the decoder 
(/best guess when missing) instead when using YUV rendering. I don't really 
understand why we would want to agree on using RGB but still keep the door open 
to more matrix combinations. All kinds of different combinations will never be 
implemented in other filters and they would require lots of work for RGB 
implementations.

Original comment by sneaker...@googlemail.com on 4 May 2012 at 6:52

[GoogleCodeExporter]

> If that's the case then why not choosing (1)? 

Because you can achieve the goals of (1) better by using (3).

With that one in a million chance, (3) could achieve untouched RGB subtitles 
and YCbCr subtitles which match the video where (1) would otherwise be 
incorrect. 

With that one in a million chance, (3) could achieve corrected RGB subtitles 
and YCbCr subtitles which match either other and the video where (1) would 
otherwise be incorrect. 

The risk is in the future, where it could become more common that (3) is 
usually correct, while (1) is usually incorrect.

> Well, "color workaround: on" has this information too,
> even if it is only implicit, because it means the same as

That's the entire point I'm trying to make. The only difference being we could 
allow yuvMatrix other than BT.601 in the future if we decide it's actually 
needed. It's all about the potential to solve future problems with corrected 
RGB output.

> script portability is lost (if source matrix is optional)

The entire reason source matrix should be optional is because it only matters 
for script portability.

At present, I honestly don't care much about script portability with corrected 
RGB output. If in the future script portability became enough of a problem that 
we wanted to solve it, we could.

> certain chains don't work. Like video decoder being set to RGB output

And certain chains wouldn't work with what you're proposing either. This isn't 
a disadvantage. Today it okay to accept that certain script behavior isn't 
allowed. The difference being that what I propose allows us to support more 
script behavior in the future on an as needed basis.

> it mixes inherent video attributes into the subtitle file

And by doing so, it simplifies YCbCr output while offering potential advantages 
for RGB output.

> correct rendering needs info about video even with the new interface, 
> the whole chain would need to pass on that information.

Of course it would, but at the same time yuvMatrix could allow solving (or at 
least identifying) problems you wouldn't be able to otherwise.

> users who do not create their files and set up their playback chains correctly
> will always have problems. Trying to work around that is futile.

Yet we could workaround this so they don't have problems with what I propose. 
Today I don't think we should, but maybe a few years into the future there 
would actually be a need to do so. 

> I don't think we want to "gradually evolve" ASS anymore. This is basically a
> one time only thing to fix a vsfilter problem, 

The this is a one-time thing is exactly why I want to add capability which 
would solve any matrix problem you could imagine into the future, even if today 
such functionality is largely unneeded (and we should leave unsupported atm). 
The ability to "gradually evolve" how we handle matrix information, while not 
affecting past scripts is an advantage. It could prevent the ASS spec from 
becoming broken again.

Tagging yuvMatrix is the best solution to VSFilter's problem with YCbCr output.

Tagging yuvMatrix has no disadvantage for RGB output with the new interface if 
we don't let it.

> the simpler the approach, the more likely it is for e.g. VLC and MPC-HC to 
implement it.

We are going to implement a simple approach.

> allowing all kinds of different source and aegisub matrices means a lot of 
work for implementations.

We aren't going to allow "all kinds of different source and aegisub matrices" 
today. In the future we could vote to allow new combinations on an as-needed 
basis. If we're lucky, we will never need to change anything from our initial 
(limited, but infinitely flexible) implementation. It's all about creating a 
future-proof solution.

> what was supposed to be a hint becomes the source of an error when remuxing

In case of rgbCorrection=false, it's an easily avoided error.

In case of rgbCorrection=true, you will always be wrong anyway.

> I don't really understand why we would want to agree on using RGB but 
> still keep the door open to more matrix combinations.

You're looking at this problem from the wrong perspective.

My proposal:
There is no harm in "keeping the door open", since we are the gatekeepers.

Your proposal:
We will be unable to "open the door" if we ever have a dire need to.

You are unable to claim my proposal is going to lead to added complexity, if at 
the same time you claim with your proposal that added complexity would never be 
needed. If you admit the added complexity from my proposal may be needed in the 
future, it means your proposal is flawed.

Original comment by cyber.sp...@gmail.com on 4 May 2012 at 10:14

[GoogleCodeExporter]

This may be just a (possibly stupid) proposal, but for those cases where a 
legacy "workaround" script (explicitly flagged, of course, otherwise, this 
example would be useless) is used in combination with a video that is already 
BT.601, is there a possibility (or necessity?) to include some kind of YUV 
matrix that creates the correct colours for this case?
I understand that normally, this won't be needed, because either the output is 
via the new interface and therefore being taken care of already or in 
combination with a BT.709 video, which would lead to xy-VSFilter only needing 
to overlay the BT.601 colours onto the BT.709 video to produce correct colours, 
but I think that in this case, the colours would have to be corrected so the 
colours that the BT.601 YUV would have been decoded to in RGB colourspace when 
using BT.709 would need to result in being used when converted to RGB using the 
video's BT.601 matrix (assuming my assumptions about YUV and different release 
versions using different matrices from posts #31 [last paragraph] and #66 are 
correct).

Original comment by TheDarkS...@gmail.com on 4 May 2012 at 10:55

[GoogleCodeExporter]

Going back to the topic of the actual SubRenderIntf specification we're 
designing, I'd propose that instead of just one specification which supports 
everything, we split the it into three feature tiers.

Tier 1 (Advanced): Untouched RGB + Complex Corrected RGB + ???

Tier 2 (Standard): Untouched RGB + Limited Corrected RGB
(madVR + xy-VSFilter + Aegisub support Tier 2)

Tier 3 (Basic): Untouched RGB only

Tier 2 is a subset of the features of Tier 1
Tier 3 is a subset of the features of Tier 2.

The availability of a Tier 3, would ease into other developers adding support 
for our interface, while at the same time Tier 2 would clearly define which ASS 
matrix functionality (of what I've currently been proposing) we are supporting. 
It would take a bit more planning to decide which other features belong in Tier 
1 & 3, but in the long run I think it would be worth it.

Doing this would also help separate decisions about when/how/why we support 
certain features in the new interface from any particular subtitle format, 
SSA/ASS or otherwise.

Original comment by cyber.sp...@gmail.com on 5 May 2012 at 1:57

[GoogleCodeExporter]

Sorry, cyber, but you want to add a lot of complexity for something which you 
suggest yourself is "one in a million chance". It's just not worth the 
headaches. Let's use the simple approach and ignore the "one in a million 
chance". Unless you can raise valid concerns (with a good example) of why we 
might come into trouble in the future, when using the simple approach. In case 
you haven't noticed, you seem to be the only one voting for explicit matrix 
tagging. Everyone else seems to prefer the simple approach.

From my point of view, as long as the video is authored correctly, the simple 
approach is all we'll ever need. There can be problems only if the video uses a 
matrix which doesn't fit to resolution based auto guessing and if on top of 
that the video bitstream doesn't tell which matrix it uses. I would consider 
such a video file broken and I would really like to refuse adding complicated 
workarounds to cater for such broken video files. As long as the video is 
authored correctly, we should even be safe for any potential new matrix in the 
future.

Original comment by mad...@gmail.com on 5 May 2012 at 7:07

[GoogleCodeExporter]

> Sorry, cyber, but you want to add a lot of complexity for something 
> which you suggest yourself is "one in a million chance".

Where exactly are you getting this idea of added complexity? What I'm 
suggesting has no added complexity, it actually makes things simpler. 
Implementation would be identical with or without yuvMatrix included as far as 
the new interface is concerned (unless you desired otherwise).

If it makes you feel better, I can clearly say we are effectively ignoring 
yuvMatrix (unless you decide otherwise) when the new interface is used. I've 
been telling you that in so many words for the last few comments...

So with this understanding that yuvMatrix wouldn't affect the new interface 
(unless madshi desires it so), what exactly is your objection?

Original comment by cyber.sp...@gmail.com on 5 May 2012 at 7:43

[GoogleCodeExporter]

As I said before, YCbCr and RGB rendering should produce the same results. If 
there's an option to do funny things with the matrix for YCbCr rendering then 
subtitle authors might be tempted to use it (they could specify a matrix 
different from the video), and if RGB rendering simply ignores that, colors 
would be different between YCbCr and RGB rendering. That is a no go. So if we 
add matrix information to the ASS files, RGB rendering has to interpret and 
handle it, too.

If we change the ASS spec, we should have a clear idea in mind how to interpret 
the ASS colors. Should they be interpreted as native YCbCr colors or native RGB 
colors? And this is a point which you're totally ignoring. You're only looking 
at how to get away with the biggest feature set for YCbCr rendering, while 
ignoring that your wishes have consequences outside of your preferred YCbCr 
renderer. If you want to change the spec you need to take a step back and look 
at the bigger picture. The spec should be independent of a specific renderer 
implementation. Your idea might work well for YCbCr rendering, but you're 
ignoring that your wishes result in added complexity for RGB rendering. Now 
there's absolutely no law that would forbid us from adding complexity for RGB 
rendering, but any added complexity should be worth the extra effort. And I 
simply don't see the added benefit here.

Don't get me wrong. I could live with interpreting the ASS colors as native 
YCbCr colors and with having explicit matrix tagging. I would also make sure 
RGB rendering would produce identical results to YCbCr rendering in that case. 
But I have to say that it is my least favorite solution. And I think at some 
point you have to bow to the majority vote, especially if you have the LAV 
Splitter and madVR devs and an Aegisub dev preferring the simpler approach. 
Let's not even mention the libass devs which have always had the opinion that 
ASS colors should be treated as native RGB colors (AFAIU).

Original comment by mad...@gmail.com on 5 May 2012 at 8:28

[GoogleCodeExporter]

> As I said before, YCbCr and RGB rendering should produce the same results.

Null argument, since as just made clear in your previous comment, you weren't 
concerned with a "one in a million chance".

> If there's an option to do funny things with the matrix for YCbCr rendering
> then subtitle authors might be tempted to use it

I find it much more likely that "subtitle authors" would set the correct matrix 
if the auto-guessed matrix is incorrect.

> (they could specify a matrix different from the video)

Of course they could, but that's like saying a user would be tempted to set 
rgbCorrection from false to true for no reason. Why would a user purposefully 
sabotage their script?

> colors would be different between YCbCr and RGB rendering.

Which wouldn't matter. If a "script author" sabotages their script, broken 
output is expected. I'm willing to accept YCbCr output being incorrect for 
borked scripts.

If the video bitstream had the matrix flagged when rgbCorrection=false, then 
the sabotage could be ignored (but I don't think we should do this).

If the video bitstream had the matrix flagged when rgbCorrection=true, then the 
sabotage could be ignored with any value other than BT.601 (if we weren't 
supporting corrections for other colorspaces).

> If you want to change the spec you need to take a step back
> and look at the bigger picture.

I am looking at the big picture, and all the way into how we could solve future 
problems which don't exist yet. You seem to only be looking at the very limited 
case which is this new interface.

You are acting like you are being forced to change the behavior of the 
interface when yuvMatrix exists, but that's not true at all. You are just being 
tempted to use all the information available (as it may be more correct).

I seem to be getting mixed messages from you. You say your preferred option is 
native RGB colors, so why don't you stick with that? That yuvMatrix exists 
shouldn't matter if your goal is interpretation as native RGB colors. You have 
a choice, and mimicking Libass behavior even when yuvMatrix exists, isn't a bad 
choice.

> Aegisub dev preferring the simpler approach

> tgoyne: Then for files with the workaround: "Color correction: BT.601,BT.709" 
> (read: "convert to BT.601, then convert to RGB assuming BT.709")

> tgoyne: I'd be more inclined to just do On for the other case rather than 
> hinting at the possibility of intentionally-wrong colorspaces other than 
BT.601

> tgoyne: BT.709 and BT.601 aren't the only possible values

> tgoyne: Writing both the matrix used by Aegisub and whether the matrix was 
intentionally wrong
> is certainly an option, of course.

> tgoyne: I don't think it's actually useful to write the video's color matrix 
to the subtitles file.

I think you may be misunderstanding him madshi. He seems to favor the 
approaches least prone to error.

Aegisub Matrix + rgbCorrection True|False

OR

rgbCorrection True|False (Force BT.601 on all videos without matrix flagged)

OR

sourceMatrix + yuvMatrix, else rgbCorrection=False

The main dilemma I'm having at this point is the expectation that every 
subtitle renderer which supports YCbCr output is going to be required extra 
complexity (required to support reading flagged video bitstreams, else risk 
being broken), reduced functionality (no automated PC-range YCbCr), and no 
capability to allow the user to override an incorrectly flagged bitstream or 
incorrect auto-guess. On top of this, it could also potentially allow more 
complex RGB corrections if needed.

Original comment by cyber.sp...@gmail.com on 5 May 2012 at 10:24

[GoogleCodeExporter]

> Null argument, since as just made clear in your
> previous comment, you weren't concerned with a
> "one in a million chance".

I'm not. But:

> Of course they could, but that's like saying a
> user would be tempted to set rgbCorrection from
> false to true for no reason. Why would a user
> purposefully sabotage their script?

I'm not talking about sabotage here. The user could set the ASS matrix to 
BT.601 for a BT.709 movie, and then choose colors which make the ASS file match 
to the BT.709 movie. He might do that accidently, and since he only checks that 
colors are correct inside of Aegisub and with VSFilter, he wouldn't notice the 
problem. But when rendering the same subtitle file with an RGB based renderer, 
suddenly colors wouldn't match (if the RGB renderer doesn't take the matrix 
information into account).

The user could also simply believe that the matrix guessed by Aegisub and 
VSFilter and madVR is incorrect and thus set a different matrix in the ASS 
file. If he does that, VSFilter YCbCr rendering will still produce matching 
colors, even if madVR still auto-guesses differently, but RGB rendering will 
produce non-matching colors.

> Which wouldn't matter. If a "script author"
> sabotages their script, broken output is expected.
> I'm willing to accept YCbCr output being incorrect
> for borked scripts.

But you're not willing to accept incorrect output with borked video files? I 
consider video files which have a non-auto-guessable matrix and which don't 
have the right matrix referenced in the video bitstream as borked.

> I am looking at the big picture, and all the
> way into how we could solve future problems
> which don't exist yet.

What problems do you have in mind that could crop up in the future? A good 
example would be helpful.

> You are acting like you are being forced to
> change the behavior of the interface when
> yuvMatrix exists, but that's not true at all.

I don't want users to come to me and complain that colors are incorrect when 
using madVR, while they're alright when using VSFilter. Is that so hard to 
understand? I want a clear definition of how colors are to be interpreted, and 
I want it to apply to all renderers, YCbCr and RGB.

> The main dilemma I'm having at this point is
> the expectation that every subtitle renderer
> which supports YCbCr output is going to be
> required extra complexity (required to support
> reading flagged video bitstreams, else risk
> being broken)

Both LAV and ffdshow video decoders already inform VSFilter about the color 
matrix. CoreAVC will add that with version 3.5, I've been told.

> reduced functionality (no automated PC-range
> YCbCr)

If there's a guarantee that having range information in the ASS file won't have 
any effect in any situation on RGB rendering (which I'm still not convinced 
about) then I've no problem with that. My main worry is that we might change 
the spec in such a way that YCbCr and RGB rendering may produce different 
output, with "correct" ASS files.

> no capability to allow the user to override
> an incorrectly flagged bitstream or incorrect
> auto-guess.

Oh, and what happens if the user does override an incorrectly 
flagged/auto-guessed matrix? Then YCbCr rendering will produce matching 
subtitle colors, but RGB rendering won't - unless RGB rendering is taking the 
matrix information into account, too.

Original comment by mad...@gmail.com on 5 May 2012 at 12:48

[GoogleCodeExporter]

> The user could set the ASS matrix to BT.601 for a BT.709 movie,
> and then choose colors which make the ASS file match to the BT.709 movie. 
> He might do that accidently, and since he only checks that colors are 
> correct inside of Aegisub and with VSFilter, he wouldn't notice the problem.

If I'm understanding you correctly:
User tags yuvMatrix=BT.601, rgbCorrection=False to script.
User opens script in Aegisub along with BT.709 video.
User does typesetting and saves script.

> But when rendering the same subtitle file with an RGB based renderer, 
> suddenly colors wouldn't match (if the RGB renderer doesn't take the
> matrix information into account).

This shouldn't occur because Aegisub always corrects (overrides) script values 
when a script is saved.

> The user could also simply believe that the matrix guessed by Aegisub 
> and VSFilter and madVR is incorrect and thus set a different matrix in 
> the ASS file. If he does that, VSFilter YCbCr rendering will still produce
> matching colors, even if madVR still auto-guesses differently, but RGB 
rendering
> will produce non-matching colors.

Assuming in this case you mean the user overrides the decoding matrix in 
Aegisub, this again wouldn't be a problem since the user would also set 
rgbCorrection=true as they were already aware/believe the auto-guess matrix 
choice was incorrect.

> But you're not willing to accept incorrect output with borked video files?

There always needs to be a way to have YCbCr subtitle output match the colors 
of any video file, borked or not.

> What problems do you have in mind that could crop up in the future? 
> A good example would be helpful.

Having a matrix other than BT.601 and BT.709 which couldn't be auto-guessed. 
I've already given you numerous examples of this in one of my previous comments.

The basic premise is anything which could cause the matrix behavior of multiple 
programs dealing with ASS scripts to vary from each other.

A simple example which I don't believe I've mentioned before, would be the 
video resolution to assume BT.709 instead of BT.601.

Or how about codec specific matrix detection? Would you cripple madVR from 
using the correct video matrix if Aegisub and/or VSFilter didn't support codec 
specific matrix detection, yet did support the correct matrix?

Let's say an uncommon matrix was encountered which madVR supports but 
xy-VSFilter didn't. The user forces BT.601 to workaround this. YCbCr output 
would match the video, but RGB output would not.

History has proven that hardly anybody is brave enough to make changes to 
VSFilter. At a certain point, xy-VSFilter project may be without a programmer 
to maintain it. This could give rise to the possibility that a brand new 
VSFilter matrix compatibility problem is born.

> Both LAV and ffdshow video decoders already inform VSFilter about 
> the color matrix. CoreAVC will add that with version 3.5, I've been told.

That doesn't change that every subtitle filter with YCbCr output would need to 
be capable of reading the bitstream matrix in order to have correct output with 
all decoders. 

> Then YCbCr rendering will produce matching subtitle colors, but RGB rendering 
won't 
> - unless RGB rendering is taking the matrix information into account, too.

RGB subtitle output is just a major pita to get correct whenever something 
unexpected occurs.

AYUV subtitle output could solve this, if it was a valid option.

Original comment by cyber.sp...@gmail.com on 5 May 2012 at 4:38

[GoogleCodeExporter]

Didn't have the time to read everything yet (#83), but why would range be a 
problem? Range is signaled in the bitstream, just like the matrix is.

Original comment by sneaker...@googlemail.com on 5 May 2012 at 4:52

[GoogleCodeExporter]

> Range is signaled in the bitstream, just like the matrix is.

In the past, I seem to remember hearing about problems with certain TV stations 
flagging everything incorrectly as fullrange.

Original comment by cyber.sp...@gmail.com on 5 May 2012 at 5:15

[GoogleCodeExporter]

I think I need to clarify something from the last few comments, since madshi 
got me confused. Have we really been debating this for almost a month now...

yuvMatrix should never be changed by the user when rgbCorrection=false. It 
should authoritatively represent the matrix used when authoring the subtitles 
100% of the time.

Only sourceMatrix (optional only when rgbCorrection=true) would be allowed to 
be changed by the user if incorrect.

Original comment by cyber.sp...@gmail.com on 5 May 2012 at 5:46

[GoogleCodeExporter]

This goes back to how when yuvMatrix (a DO NOT TOUCH value) is left unchanged 
as it should be, YCbCr output and RGB output should always match as of today.

I original listed sourceMatrix as optional because it's a value we shouldn't 
support unless it's needed in the future.

Original comment by cyber.sp...@gmail.com on 5 May 2012 at 6:11

[GoogleCodeExporter]

[deleted comment]

[GoogleCodeExporter]

There is also no need for yuvMatrix to be passed to madVR over the new 
interface, since it should always be assumed to match the matrix Aegisub, 
xy-VSFilter, and madVR would normally use.

If the user changes yuvMatrix (a DO NOT TOUCH value) when rgbCorrection=false, 
the script would be invalid/sabotaged, causing YCbCr output to be incorrect and 
not match the video, but RGB output still correct and match the video. 

If the user changes yuvMatrix (a DO NOT TOUCH value) when rgbCorrection=true, 
the script would be invalid/sabotaged, but since the only supported value is 
BT.601, the sabotage would be ignored and treated as BT.601 anyway for both 
YCbCr and RGB output.

I don't have a problem with either case. Users will learn over time not to mess 
with these values.

Original comment by cyber.sp...@gmail.com on 5 May 2012 at 7:05

[GoogleCodeExporter]

In conclusion, the "yuvMatrix + rgbCorrection" method behaves identically to 
"rgbCorrection only" for RGB output. As of today, RGB output will never be 
incorrect when YCbCr is correct.

It also behaves identically to sneakerger's proposal, as I initially claimed. 
YCbCr output gets simplified, without any negative effect on RGB output.

If in the future we run into some problems like those I mention in Comment #83 
(and other previous comments), we could make use of sourceMatrix (which we 
defined as optional) in the new interface.

_____
Well I'm off to bed, since I was up all night and it's now past noon...

Original comment by cyber.sp...@gmail.com on 5 May 2012 at 7:31

[GoogleCodeExporter]

For the record, if we do decide to use explicit matrix tagging, I absolutely 
and very strongly vote for only storing the Aegisub matrix into the ASS file. 
No other matrix, please, and no "rgbCorrection" field, either. I find the 
combination of "yuvMatrix + rgbCorrection" confusing and superfluous.

> The basic premise is anything which could cause
> the matrix behavior of multiple programs dealing
> with ASS scripts to vary from each other.

But if multiple programs "auto-guess" different matrixes, then isn't that 
either a broken video file or a bug in one of the programs? I don't think we 
have to make the first work, and the second should be easy enough to fix.

> Or how about codec specific matrix detection? Would
> you cripple madVR from using the correct video matrix
> if Aegisub and/or VSFilter didn't support codec
> specific matrix detection, yet did support the correct
> matrix?

First of all, this would most likely happen with a new codec and a new decoder, 
correct? I would hope that the new decoder is written well enough to output the 
matrix information via the MS recommended information fields. If that doesn't 
work out, VSFilter could still fall back to asking the matrix from the video 
renderer. Of course there's a chance all of that fails, but I'm wondering how 
probable that really is. Most new video codecs are supported rather quickly by 
libav/ffmpeg, so I think for new video codecs probably LAV will be the 
preferred video decoder for all new video codecs for years to come, and LAV is 
already capable to inform VSFilter about which matrix needs to be used. All the 
work for that is done by the libav/ffmpeg guys, Hendrik doesn't need to do much 
to add support for new video codecs. So I don't see a big danger of LAV getting 
out of date anytime soon.

Anyway, I think we've discussed this in circles and there will be no new 
arguments, anymore. So here comes my final word on this:

(1) I can live with explicit matrix tagging, as long as the Aegisub matrix is 
the only new information written to the ASS file (except maybe for levels 
information). When using explicit matrix tagging, please stay away from adding 
additional information (like "rgbCorrection"). If you add 2 new fields, 
everybody will have to wonder about how the 2 new fields influence each other. 
So having only 1 new field is easier to understand.

(2) Although I could live with explicit matrix tagging, I believe it's not the 
right solution because of multiple different reasons (which I've mentioned 
scattered over the last few hundred posts). I would prefer either native RGB 
interpretation or always using the BT.601 workaround.

Not sure who can make the final decision. If the decision is made in any sort 
of rational way (democratic vote perferred) then I'll accept the result and 
support it.

'nough said...

Original comment by mad...@gmail.com on 5 May 2012 at 8:43

[GoogleCodeExporter]

> For the record, if we do decide to use explicit matrix tagging,
> I absolutely and very strongly vote for only storing the Aegisub 
> matrix into the ASS file. No other matrix, please, and no "rgbCorrection"
> field, either. I find the combination of "yuvMatrix + rgbCorrection" 
> confusing and superfluous.

The problem is yuvMatrix (Aegisub matrix) is semi-flawed when used on its own.

Use of yuvMatrix + "rgbCorrection=false" has a distinct advantage in terms of 
script portiblity.

> But if multiple programs "auto-guess" different matrixes, then isn't that 
either a broken video file 
> or a bug in one of the programs? I don't think we have to make the first work,
> and the second should be easy enough to fix.

The concern is basically a bug, or possibly just a disagreement if said bug is 
actually worth fixing.

A perfect example is right now, with xy-VSFilter is on haitus for 5 months 
already. Who is going to fix a bug?

Maybe the bug never gets fixed. How would madVR handle that? Display subtitles 
with incorrect colors or purposefully display the video with incorrect colors? 
Added complexity and script information would be needed to result such a 
problem for madVR.

Overall it's not too bad with keeping just three programs in sync, but if more 
subtitle editors, subtitle filters, and video renderers supporting the new 
interface emerge, keeping everything in sync becomes exponentially more 
difficult.

That's why I've been saying it a good idea to have additional information in 
the script for troubleshooting in the future, even if as of right now 
everything should work just fine with a lot of that extra information 
assumed/guessed. If we're lucky we will never run into problems, but if we do 
run into problems, they will be much easier to fix if we already have the 
necessary tools in the ASS spec.

> Of course there's a chance all of that fails, but I'm wondering how probable 
that really is.

How good is your crystal ball?

I've personally had notoriously bad luck through my life (where everything 
which could go wrong does go wrong...). Eventually I got into the habit of 
always planing how to deal with anything which could go wrong, even worse-case 
scenario, so I always stay ahead. Better to be safe than sorry.

I would like a minimalistic solution as well, but the risk of needing a complex 
solution makes me desire to go above and beyond in defining information which 
may be useless to us now, in order to make the future easier. At this point I'm 
suspecting we will run into a problem eventually, but for the time being we may 
keep things a simple as possible, with an easy transition into a more complex 
solution if needed. 

That goes back to what I suggested in Comment #77.

> I would prefer either native RGB interpretation

What I'm proposing would be a native RGB interpretation, as of today.

If we run into problem in the future, only then would it change to a YCbCr 
interpretation. 

If we never run into problems, it stays a native RGB interpretation forever.

Original comment by cyber.sp...@gmail.com on 6 May 2012 at 6:09

[GoogleCodeExporter]

> The problem is yuvMatrix (Aegisub matrix) is
> semi-flawed when used on its own.
> Use of yuvMatrix + "rgbCorrection=false" has
> a distinct advantage in terms of script portiblity.

I don't understand. Why? Would VSFilter in YCbCr rendering mode make use of 
"rgbCorrection"? Makes no sense to me right now.

> What I'm proposing would be a native RGB
> interpretation, as of today.

Totally wrong. But I've no fun to repeat the same arguments all over again.

Original comment by mad...@gmail.com on 6 May 2012 at 6:47

[GoogleCodeExporter]

> I don't understand. Why? Would VSFilter in YCbCr rendering mode 
> make use of "rgbCorrection"? Makes no sense to me right now.

You don't understand because I'm not referring to YCbCr output.

The distinct advantage of having "rgbCorrection=false" is for RGB output.

> Totally wrong. But I've no fun to repeat the same arguments all over again.

It's actually totally correct, but I have no desire to re-explain again either. 
Read my previous posts again if you want to understand.

What I'm proposing would behave identically (assuming the user doesn't sabotage 
their script), even if yuvMatrix was missing.

RGB output would always be correct and match the script colors, as of today.

Original comment by cyber.sp...@gmail.com on 6 May 2012 at 7:11

[GoogleCodeExporter]

[deleted comment]

[GoogleCodeExporter]

> The distinct advantage of having "rgbCorrection=false"
> is for RGB output.

Do you need this for the conventional VSFilter way of rendering, e.g. with 
native RGB videos? Or are you just trying to "help" RGB subtitle renderers 
(like the new subtitle interface) by introducing "rgbCorrection"?

> It's actually totally correct

You're not even trying to get my point. At least I spent some serious time 
trying to get yours. It's too bad you're not giving me the same courtesy.

Original comment by mad...@gmail.com on 6 May 2012 at 8:00

[GoogleCodeExporter]

> Do you need this for the conventional VSFilter way of rendering, 
> e.g. with native RGB videos? 

> I feel like all the energy I put into this discussion is totally wasted on 
you.
> You're not even trying to get my point. At least I spent some serious time 
> trying to get yours. It's too bad you're not giving me the same courtesy.

I already fully understand your point, but you are still misunderstanding mine.

It won't be a problem if we (Aegisub + xy-VSFilter + madVR) implement the 
solution to work identically even if yuvMatrix didn't exist.

Anything else would be unsupported and wrong everywhere, which again isn't a 
problem. If someone wants correct output, without using Aegisub, they need to 
follow the rules we set.

Possibilities today if implementing what I propose:

Both RGB and YCbCr are correct (RGB & YCbCr match)

RGB is wrong and YCbCr is wrong (RGB & YCbCr match)

RGB is correct and YCbCr is wrong (RGB & YCbCr don't match)

Not a possibility today if implementing what I propose:

RGB is wrong and YCbCr is correct (Aegisub could not create such a possibility)

Please tell me you understand this... If you don't, please tell me exactly what 
part you don't understand and I'll explain it. I would really like to get you 
on-board, so we can just move forward.

What I'm saying is correct. If you believe it's incorrect, you are 
misunderstanding the facts.

Original comment by cyber.sp...@gmail.com on 6 May 2012 at 8:24

[GoogleCodeExporter]

[deleted comment]

[GoogleCodeExporter]

> The distinct advantage of having "rgbCorrection=false"
> is for RGB output.

Do you need this for the conventional VSFilter way of rendering, e.g. with 
native RGB videos? Or are you just trying to "help" RGB subtitle renderers 
(like the new subtitle interface) by introducing "rgbCorrection"?

> I already fully understand your point

Nope, not at all. What I really want is a spec which clearly defines how ASS 
color values are to be interpreted and I want it to be totally independent of 
the implementation.

Can you in one short paragraph sum up how any subtitle renderer (regardless of 
whether it's rendering in YCbCr or RGB) has to interpret ASS color values? You 
should be able to, if your new spec has any merit.

Original comment by mad...@gmail.com on 6 May 2012 at 8:38

[GoogleCodeExporter]

Here's how I would define the new spec for the various options we have. These 
are short paragraphs which could be printed into e.g. the Aegisub manual/help:

-------

(1) explicit matrix tagging:

"ASS color values should be interpreted as native YCbCr colors, but they're 
stored in the ASS file in RGB format. As a first step, every subtitle renderer 
should apply the "yuvMatrix" to get the original YCbCr colors. The YCbCr 
subtitles should then be blended into the video before converting the video to 
RGB."  *)

(2) native RGB:

"ASS color values should be interpreted as native RGB colors. The RGB subtitles 
should be blended into the video after converting the video to RGB."  *)

(3) compatability mode for old ASS files:

"ASS color values should be interpreted as native YCbCr colors, but they're 
stored in the ASS file in RGB format. As a first step, every subtitle renderer 
should apply the BT.601 matrix to get the original YCbCr colors. The YCbCr 
subtitles should then be blended into the video before converting the video to 
RGB."  *)

*) A different technique to render the subtitles is allowed, as long as it ends 
up with the same final color output.

-------

All these 3 paragraphs are very clear and very easy to understand, which is one 
of the most fundamental requirements when writing/changing a spec, IMHO. Of 
course, if we use native RGB colors, a YCbCr renderer would come into trouble 
because it can't render the subtitles after the video is converted to RGB. So 
it has to use tricks to achieve the same effect (by using the inverse of the 
matrix used in the playback chain to convert the video to RGB). On the other 
hand, if we use native YCbCr colors, the RGB renderer gets into trouble, 
because it can't draw the subtitles before the video is converted to RGB. So it 
also has to use tricks to achieve the same effect (by converting the subtitles 
to RGB with the same matrix used to convert the video to RGB).

In case of explicit matrix tagging, as you can see, there's no need for an 
"rgbCorrection" field at all.

-------

Totally different thought: Have you thought about leaving the YCbCr video 
stream untouched and instead providing an additional RGBA output pin in 
xy-vsfilter? That should theoretically work with VMR/EVR, I think. It might 
even make xy-vsfilter compatible to DXVA video decoding. But I'm not 100% sure.

Original comment by mad...@gmail.com on 6 May 2012 at 9:29