Ability to tag Script Layout Resolution to the ASS header

[GoogleCodeExporter]

Motivation for this request:
The ability to unambiguously define Script behaviour independently from Video 
Resolution, including the knowledge of whether PAR Compensation should be used 
for a given script

Description of the feature or improvement:
Recently, in Issue #6, there was a patch introduced to enable libass to know 
about the video size that the subtitles normally would be rendered at.  At 
xy-VSFilter (here: 
http://code.google.com/p/xy-vsfilter/issues/detail?id=97#c22), I asked if there 
was some interest in tagging the Script Layout Resolution to the ASS file 
header in order to be able to abstract the script from the video it is used 
with.
The Layout Resolution would override the Video Resolution when the respective 
tags are present, which would mean that the same subtitle script can be reused 
across different video resolutions.  Also, a script of e.g. 720x480 Script 
Resolution would have been ambiguous previously (due to VSFilter's PAR 
Compensation feature), but when the Layout Resolution is tagged to e.g. 
853x480, then the values will be scaled from 720x480 to that resolution the 
"classical VSFilter way" (keep 100% x/y ratio, don't scale blur values, ...) 
and from there on to the output resolution the normal way (scale everything).  
Similarly, if the Layout Resolution had been 720x480 as well, then the script 
would have been scaled from 720x480 to the output resolution directly.  When no 
Layout Resolution header field is present, subtitles would be rendered normally.

Since the way to signal the video resolution to libass (at least that's how I 
understand Issue #6) already is there, I hope that this change will at least be 
considered.  That being said, I'd like to know if there is a general interest 
in this feature.  Also, if you've got any proposals, for example about the 
naming of these tags, I'd like to know about it as well.

Original issue reported on code.google.com by TheDarkS...@googlemail.com on 14 May 2013 at 4:07

[GoogleCodeExporter]

>Layout Resolution

Uh what, isn't that PlayResX and PlayResY?

> which would mean that the same subtitle script can be reused across different 
video resolutions

I don't follow. This worked for years with libass. (Minus some bugs about font 
scaling which were recently fixed.)

>Since the way to signal the video resolution to libass (at least that's how I 
understand  Issue #6 ) already is there

That's only a hint, and doesn't have to match the actual video resolution. (IMO 
it would be a pain on the player level to do this correctly if the video can 
resize during playback and such.) Currently it affects the blur scale only. As 
far as I understand, you can get video-independent scaling in libass if you do 
not set the video resolution with ass_set_storage_size().

So, we just recently added this, and now you want a way to disable this via 
script? That's quite funny. Or did I misunderstand something?

Related quote from #libass:
> <ChrisK4> TL;DR: Introduce new tags because VSfilter broke the old ones?

Original comment by nfxjfg@googlemail.com on 16 May 2013 at 9:27

[GoogleCodeExporter]

> Related quote from #libass:
> > <ChrisK4> TL;DR: Introduce new tags because VSfilter broke the old ones?
I'm afraid this it about it.

> Uh what, isn't that PlayResX and PlayResY?
Unfortunately, it's not, at least for VSFilter:  For example, VSFilter doesn't 
scale \blur values at all - which means that a 1280x720 script at 1920x1080 
video will have its blur values "scaled" (by the video renderer) from 1920x1080 
to whatever output resolution.  In contrast, libass would scale the \blur from 
1280x720 to whatever output resolution directly (not taking into account Issue 
#6).  Also, without its PAR Compensation feature, VSFilter scales from whatever 
Script Resolution to whatever Video Resolution so that the fonts' scale 
(without \fscx/y) is always 1:1 x/y.

> I don't follow. This worked for years with libass. (Minus some bugs about 
font scaling which were recently fixed.)
> That's only a hint, and doesn't have to match the actual video resolution.
I have obviously misunderstood Issue #6's fix, then.  I was under the 
impression that the player now may send the unscaled video's resolution to 
libass so libass can calculate the scaled \blur values from that.  I'm curious: 
 How did you solve the \blur "mis-scaling" then?

> IMO it would be a pain on the player level to do this correctly if the video 
can resize during playback and such.
Again, my misunderstanding:  I thought these fields would be static (save for a 
change in the video's resolution).

> As far as I understand, you can get video-independent scaling in libass if you
> do not set the video resolution with ass_set_storage_size().
I am aware of this.  However, I thought it is the player's decision whether to 
set this field?

> So, we just recently added this, and now you want a way to disable this via 
script?
> That's quite funny. Or did I misunderstand something?
I hope you did misunderstand something:  My aim is not to disable this 
functionality, but to set the values not from the video resolution that the 
player sends to libass, but to use values that are specified in the script 
itself (when available).

Much of my original text was written based on how I misunderstood Issue #6 
making use of the video's original resolution, thereby making the script 
somewhat dependent on the video resolution.  My aim is to make the scaling 
completely independent from the video resolution.

Original comment by TheDarkS...@googlemail.com on 16 May 2013 at 1:36

[GoogleCodeExporter]

>Also, without its PAR Compensation feature, VSFilter scales from whatever 
Script Resolution to whatever Video Resolution so that the fonts' scale 
(without \fscx/y) is always 1:1 x/y.

libass always leaves fonts at the PAR requested by the video player. That is, 
even if PlayResX/PlayResY is different from video_width/video_height, fonts are 
not "stretched" into any direction. It's up to the player to set a PAR, and in 
fact mplayer2 and mpv use this to implement the --ass-vsfilter-aspect-compat 
option (while libass doesn't have any "native" support for this option).

The final font size is determined by PlayResY, and the height of the output 
frame. The font is "stretched" in X direction by the player specified PAR.

>I have obviously misunderstood  Issue #6 's fix, then.  I was under the 
impression that the player now may send the unscaled video's resolution to 
libass so libass can calculate the scaled \blur values from that.

No, you are correct. Maybe I wasn't very clear on this. But as long as you 
don't call ass_set_storage_size(), scaling should be completely independent 
from the video resolution (except maybe for player PAR correction). There's 
some crap about the "ScaledBorderAndShadow: no" case though (IMO completely 
broken in libass, it would be better to disable handling of this header.)

>> As far as I understand, you can get video-independent scaling in libass if 
you
>> do not set the video resolution with ass_set_storage_size().
>I am aware of this.  However, I thought it is the player's decision whether to 
set this field?

It is. I guess you are saying that the script should be able to request that 
the renderer ignore the original video size?

>My aim is to make the scaling completely independent from the video resolution.

Yes. Issue #6 removed that in favor of vsfilter compatibility, and that is what 
I find funny. I mean, seriously... back and forth?

Original comment by nfxjfg@googlemail.com on 16 May 2013 at 2:15

[GoogleCodeExporter]

I think you didn't misunderstand issue #6. I'm sure you know the following, but 
let me recap. (This may also be useful as an explanation for anyone else who 
might happen on this thread...)

VSFilter is traditionally used in the following chain: decoder -> VSFilter -> 
renderer, where rescaling (including aspect ratio correction for anamorphic 
content) and colorspace conversion is done in the renderer. This is mirrored by 
mplayer2 with vf_ass. But mplayer2's EOSD, mpv's OSD and MPC-HC's subtitle 
rendering interface all paint subtitles on the rendered surface, which is 
already rescaled and converted to RGB.

Indeed, traditional VSFilter doesn't scale \blur and change aspect ratios at 
all. But in the de facto standard filter chain this means that \blur is 
ultimately scaled by the video renderer from the video storage resolution (in 
which \blur is rendered by VSFilter) to the video rendering resolution. 
Similarly, subtitle aspect ratios are changed by the video renderer when it 
corrects anamorphic content.

My fix to issue #6 is about emulating this in the other filter chain. The 
player tells libass both the video rendering resolution and the video storage 
resolution, and libass acts as if it first rendered the subtitles in storage 
resolution and then rescaled them to the rendering one.

(Except anamorphic AR correction is done separately, this time to somewhat 
preserve compatibility with scripts designed for older libass. Do they even 
exist? Maybe it's time to drop the --ass-vsfilter-aspect-compat option and fold 
anamorphic AR correction into ass_set_storage_size?)

Before this change, libass had completely resolution-independent rendering of 
scripts with ScaledBorderAndShadow:yes. But this is no longer the case when 
ass_set_storage_size isn't called: as I pointed out in 
http://code.google.com/p/libass/issues/detail?id=6#c9, the \blur scale factor 
defaults to 1. This can be changed though. The idea of such a default was to 
effectively provide better VSFilter compatibility until players are updated to 
actually use ass_set_storage_size.

Now we have a similar scheme in xy-VSFilter. So far all of this has focussed on 
compatibility, and no new fields have been necessary.

Now, you're suggesting adding a way to make scripts completely 
resolution-independent, right? In other words, scaling \blur in addition to 
ScaledBorderAndShadow:yes. Is there any other affected feature that I forget? 
When I think about it, indeed, there's currently no (VSFilter-compatible) way 
to make sure \blur is always scaled along with everything else, so this sounds 
like a good idea. But as wm4 has said on the xy-VSFilter tracker, a single 
Boolean flag that does just this should be enough.

Original comment by chortos@inbox.lv on 16 May 2013 at 2:58

[GoogleCodeExporter]

>Except anamorphic AR correction is done separately, this time to somewhat 
preserve compatibility with scripts designed for older libass. Do they even 
exist? Maybe it's time to drop the --ass-vsfilter-aspect-compat option

AFAIK different versions of VSFilter handled this differently. Maybe it doesn't 
have much to do with VSFilter itself, but the way VSFilter was used, and then 
there's mpc-hc's ISR, which uses a modified VSFilter. In other bug, it's a big 
fucking mess. Also the reason why trying to achieve perfect compatibility is 
probably pointless.

Also I like this option, I use it together with overriding default styles.

>and fold anamorphic AR correction into ass_set_storage_size?

But this is highly dependent on how the video is rendererd.

Original comment by nfxjfg@googlemail.com on 16 May 2013 at 3:31

[GoogleCodeExporter]

> Also the reason why trying to achieve perfect compatibility is probably 
pointless.

The impossible just takes a little longer.

>> and fold anamorphic AR correction into ass_set_storage_size?
> But this is highly dependent on how the video is rendererd.

We're just emulating the traditional VSFilter chain. Conceptually, we plan to 
render the image just like traditional VSFilter does and apply resolution-based 
corrections on top of that.

* When libass is used for subtitle rendering through vf_ass, it has rendering 
res = storage res (= real video storage res), so no correction is performed.
* When libass is used for subtitle rendering through EOSD etc., it has a 
correct-aspect rendering resolution and an anamorphic storage resolution, so it 
stretches the rendering accordingly, just like the video stream is stretched.
* When libass is used for actual OSD rendering through EOSD, it has rendering 
res = storage res (= real video rendering res), so no correction is performed.
* When libass is used for actual OSD rendering through vf_ass, it has an 
anamorphic rendering resolution and a correct-aspect storage resolution, so it 
condenses the rendering accordingly to compensate for the stretching that will 
follow.

The PARs mplayer clones set are all calculated just like this anyway. So why 
have the players calculate them and libass pretty much ignore some of the 
information it has when we can have libass calculate them from this information?

Original comment by chortos@inbox.lv on 16 May 2013 at 4:12

[GoogleCodeExporter]

> Is there any other affected feature that I forget?
Well, from Layout Resolution and Output Resolution, you can also get the Font 
PAR to use with the scaled output. Because the required information is already 
in the script, the Player no longer needs to request a PAR when those fields 
are present. In this case, I believe the PAR that the Player requested can 
safely be ignored (it could simply discard that information, or add a switch to 
force libass to override the script values, though I don't see much use in 
that). In fact, one of the uses that I see for this feature is that with 
properly flagged scripts, people can no longer mess up their configuration of 
whether to use VSFilter's PAR Compensation (or as mentioned above, mplayer2's 
or mpv's --ass-vsfilter-aspect-compat option) or not.

> >and fold anamorphic AR correction into ass_set_storage_size?
> But this is highly dependent on how the video is rendererd.
I don't follow. For example, say you've got a DVD-resolution video at 16:9 and 
the header tells you to use 853x480 as the Layout Resolution. This means that 
if you use an Output Resolution of 1920x1080, then you should be using (well, 
almost be using, save for rounding errors) 1:1 PAR for fonts, [[derived from 
the ratio between Layout and Output resolution]]. If you were to render the 
script at 720x480 (the original resolution), then you know to squish everything 
on the x-axis to about 84.4% of the original width. If the Video Renderer 
decides to zoom the video to 2000x1080 and crop off the left and right side, 
then shouldn't it just request an Output Resolution of that? Meaning that there 
would be no problem regarding PAR (it could just be calculated for the new 
Output Resolution again).
In other words, the Layout Resolution would automatically fill the 
"ass_set_storage_size" fields with its values, and libass would render to this 
resolution at 1:1 Font PAR. From there, it would scale naturally to the 
requested Output Resolution, without concerns about Font PAR or whether or not 
to scale the \blur values and such. Meaning that from Layout Resolution 720x480 
to Output Resolution 853x480, it would stretch everything to about 118.5% of 
its original width.

Original comment by TheDarkS...@googlemail.com on 18 May 2013 at 10:56

[GoogleCodeExporter]

I know I'm a little bit late to the discussion, but here are my thoughts.

The most severe problem with this is that the introduction of yet another 
scaling factor is not necessary. PlayRes just needs to be handled with 
semantics that make more sense. We did have scaling with (mostly) reasonable 
semantics in libass, back in 2008, but this was abandoned due to VSFilter 
incompatibilities. ASS is enough of a mess - please let's not complicate it 
even more. :) If anything, a flag or something should be added to switch to 
better semantics, like nfxjfg proposed.

Original comment by g...@chown.ath.cx on 19 May 2013 at 1:52

[GoogleCodeExporter]

> The most severe problem with this is that the introduction of yet another 
scaling factor is not necessary. PlayRes just needs to be handled with 
semantics that make more sense.

Well, it seems the vsfilter-xy guys want both new and old semantics, while 
still making sure that new scripts work on old scripts. 

See http://code.google.com/p/xy-vsfilter/issues/detail?id=97

> So why have the players calculate them and libass pretty much ignore some of 
the information it has when we can have libass calculate them from this 
information?

You're probably right. But at least that requires only one additional scalar to 
pass around (so just display PAR + video frame PAR, instead of display size and 
video frame size), and I wonder what I'd set the display size to to compensate 
for monitor PAR.

Original comment by nfxjfg@googlemail.com on 19 May 2013 at 1:23

[GoogleCodeExporter]

So, is there any conclusion?

Original comment by nfxjfg@googlemail.com on 29 May 2013 at 5:48

[GoogleCodeExporter]

> Well, it seems the vsfilter-xy guys want both new and old semantics, while 
still making sure that new scripts work on old scripts. 

Hmm, this kind of makes sense, but the overall semantics end up being very 
complex. I'm not completely against the "layout resolution" property, but I'd 
like to be very careful with any additions to ASS.

I don't know of any conclusion reached so far. I just skimped over the 
xy-vsfilter bug again and I'm not even sure anymore what their stance is. :)

Original comment by g...@chown.ath.cx on 30 May 2013 at 12:33

[GoogleCodeExporter]

There isn't any conclusion yet, unfortunately. Our debate kind of reached 
stalemate, so we decided to take a break.

Cyberbeing sees this new feature as an aid for adapting old scripts to new 
video resolutions. Everyone else, it seems, sees it as a way to make new 
scripts scale naturally to arbitrary video resolutions now and in the future, 
sort of an opt-in VSFilter bug fix.

This makes cyberbeing worry about compatibility with old renderers, similarly 
now and in the future. Now, his version allows creating new scripts whose 
script resolution differs from the resolution of the video they're released 
together with, having them scale naturally in the future and yet be compatible 
with old renderers upon release. With a simple Boolean sane-scaling switch the 
video resolution would be ignored, but with his version it can be manually 
copied into the new header. In the long term, he's afraid someone will come up 
with the bright idea of redefining the simple Boolean flag to mean something 
more backwards-incompatible than just "do scaling in a sane way", like the 
doctype declaration in HTML was redefined to enable a "standards mode" with a 
wide variety of changes from the default "quirks mode".

Indeed, since he's worried about old renderers and since the new header, when 
actually used, would break compatibility regardless of its form, it seems he 
quite likes the idea of not doing anything at all.

Now, I don't agree with him. But I hope this is a reasonably accurate summary 
of his opinion.

Original comment by chortos@inbox.lv on 30 May 2013 at 2:29

[GoogleCodeExporter]

Over at the xy-VSFilter issue 
(http://code.google.com/p/xy-vsfilter/issues/detail?id=97#c53), cyberbeing 
proposed that the Layout field be "semi-Boolean" by having values to describe 
if the video's or the script's native resolution shall be used for Layout, but 
with a WxH "emergency override" that can be used to directly specify the Layout 
Resolution for the purpose of fixing old scripts.

Original comment by TheDarkS...@googlemail.com on 7 Jun 2013 at 10:52

[GoogleCodeExporter]

So, are there any opinions on the
Layout: Compatible | Layout: PlayRes | Layout: WxH
structure that cyber proposed?

Original comment by TheDarkS...@googlemail.com on 29 Jun 2013 at 12:55

[GoogleCodeExporter]

Too complex if you ask me.

IMO layout dependent scaling already works. You just havr to decide what to do 
with an aspect mismatch between video and script. There are two solutions:
1) stretch the script resolution to video width
2) keep the script aspect ratio (even for \pos), so if the video width is 
smaller than script width at same heights, some \pos text might be cropped

Reusing a script on a video with different aspect ratio doesn't really make 
sense, so either way is fine. Sorry if I'm being ignorant about the previous 
discussion.

Original comment by nfxjfg@googlemail.com on 29 Jun 2013 at 1:08

[GoogleCodeExporter]

Thank you for the answer.  I'm not sure what you mean by stretching or keeping 
Script AR even for \pos tags, so I'll assume you were talking about Font PAR in 
libass.  My understanding is as follows:
- Layout: Compatible (or "video" or whatever you'd like it called) means that 
the Font PAR at the video's resolution is 1:1, so the Font PAR for the Output 
Resolution can be calculated from that.  Other than that, the \blur values (and 
other PlayRes-independent tags too) should be scaled from that resolution to 
Output Resolution.
- Layout: PlayRes (or "script" or whatever you'd like it called) means that the 
Font PAR at Script Resolution is 1:1 and that all values should be scaled 
directly from there to the Output Resolution.
- Layout: 853x480 (or any other resolution) means that at 853x480, the Font PAR 
is 1:1 and that this resolution should be treated as if it was the video's 
resolution and the field were set to use Video Resolution for Layout.
I believe this mode will indeed be useful, for example with anamorphic video 
and old scripts that were created to be used in combination with VSFilter's PAR 
Compensation feature but have a Script Resolution of 720x480.  Currently, if I 
understand things correctly, those scripts can be rendered correctly with 
libass by specifying a different Font PAR for the Output Resolution, while in 
VSFilter one needs only enable PAR Compensation.  However, in this case, 
neither the "PlayRes" nor the "Compatible" value can be used because the Layout 
Resolution (which also means a 1:1 Font PAR at that resolution) is neither of 
the two.

Original comment by TheDarkS...@googlemail.com on 29 Jun 2013 at 3:06

[GoogleCodeExporter]

>I'm not sure what you mean by stretching or keeping Script AR even for \pos 
tags, so I'll assume you were talking about Font PAR in libass.

No. Font PAR should always match monitor par (so, 1:1). I know that vsfilter + 
anamorphic require stretching fonts. There is an unfortunate hack for this in 
mplayer2/mpv, but fortunately this behavior can be disabled.

I was talking about how the \pos tag is mapped to video coordinates.

>- Layout: Compatible (or "video" or whatever you'd like it called) means that 
the Font PAR at the video's resolution is 1:1, so the Font PAR for the Output 
Resolution can be calculated from that.  Other than that, the \blur values (and 
other PlayRes-independent tags too) should be scaled from that resolution to 
Output Resolution.

Sounds perfect, especially if it's just "Layout: Compatible", and no additional 
resolution or anything is specified. Not much would change for libass. It'd 
have to export the header to the application (so that it disable the damn 
aspect hack), and it'd have to ignore ass_set_storage_size().

It still has to properly define what happens to \pos if script and frame aspect 
mismatch.

>- Layout: PlayRes (or "script" or whatever you'd like it called) means that 
the Font PAR at Script Resolution is 1:1 and that all values should be scaled 
directly from there to the Output Resolution.

OK, that's the old behavior. It implies that the vsfilter aspect hack for 
anamorphic video is active.

- Layout: 853x480 (or any other resolution) means that at 853x480, the Font PAR 
is 1:1 and that this resolution should be treated as if it was the video's 
resolution and the field were set to use Video Resolution for Layout.

I don't understand what this is for. The fact that there is yet another 
resolution (in addition to playres, cropped frame res, full screen res, video 
res, whatever res) is confusing at best. So what's the point: reusing old 
scripts? But then you can convert them in some way (write a converter is there 
isn't one yet). Maybe this would warrant the addition of a font par header 
(which effectively puts all fonts under a certain \fscx value), but not a 
resolution.

Original comment by nfxjfg@googlemail.com on 29 Jun 2013 at 3:29

[GoogleCodeExporter]

> I was talking about how the \pos tag is mapped to video coordinates.
But isn't the \pos tag always stretched/squeezed from Script Resolution to 
Output Resolution without AR concerns anyway?  I can't imagine a situation 
where not stretching it to Output Resolution but instead to the AR-keeping 
Output Resolution would be beneficial...

> Font PAR should always match monitor par (so, 1:1). I know that vsfilter + 
anamorphic require stretching fonts.
> There is an unfortunate hack for this in mplayer2/mpv, but fortunately this 
behavior can be disabled.
I don't understand.  The font stretching you mention is only needed because the 
Font PAR at the video's resolution (I'm assuming a 720x480 video with 32:27 
video PAR here) is not 1:1, so the Dialogue Sytle uses an x-scale of ~84% to 
compensate for that (which means that when it's scaled to e.g. 853x480, and the 
fonts are created at 853x480 with a Font PAR of 32:27, the output will look 
normally because the x-scaling compensates for the Font and Video PAR.
If in this case, a script doesn't compensate, it should indeed use a Font Par 
of 1:1, but in this case it has likely been designed for VSFilter's PAR 
Compensation feature (which aims to compensate for such things automatically).
Or am I missing something?  I'm not sure here... maybe the Font PAR is 
specified for the Video Resolution instead of the Output Resolution?

> >- Layout: Compatible (or "video" or whatever you'd like it called)
> It'd have to export the header to the application (so that it disable the 
damn aspect hack),
> and it'd have to ignore ass_set_storage_size().
Again, I think I may be missing something.  Isn't ass_set_storage_size() the 
field that tells libass the resolution of the video?  If so, why do you need to 
disable it if you need the Video Resolution?
Also, what Aspect Hack?  If my guess is right, can't you just ignore the Font 
PAR value specified by the calling application and use the PAR calculated from 
the Video Resolution / Output Resolution ratio instead with this header value?
> It still has to properly define what happens to \pos if script and frame 
aspect mismatch.
In that case, I'd say simply scale the \pos values without caring about AR 
mismatch.  This seems the most logical to me, I think this is what VSFilter has 
always done and I believe it's also what libass does.

> >- Layout: PlayRes (or "script" or whatever you'd like it called)
> It implies that the vsfilter aspect hack for anamorphic video is active.
Again, what hack?  Or is my guess from above right and the Font PAR is not 
specified for the Output Resolution but for the Video Resolution?

> - Layout: 853x480 (or any other resolution)
> The fact that there is yet another resolution (in addition to [...] video 
res, whatever res) is confusing at best.
I don't understand.  With Layout:  853x480, the Video Resolution is absolutely 
irrelevant, as the Layout Resolution completely replaces the Video Resolution.  
Just pretend that the Layout Resolution _is_ the Video Resolution...
> So what's the point: reusing old scripts?
Pretty much, yes.  Though this also aims to take care of e.g. a 720x480 script 
that was used with a 1280x720 video that you now want to use on a 720x480 video 
(or a 1920x1080 video) for example.
> But then you can convert them in some way (write a converter is there isn't 
one yet).
One of my problems (aside from being unable to code) is that I wouldn't exactly 
know how to convert e.g. 3D-rotations to either Script or Video Resolution for 
such scripts.
> Maybe this would warrant the addition of a font par header, but not a 
resolution.
Hmm, I'm not sure...  I can see a resolution being useful here, as e.g. \blur 
values would also be scaled from that "fake" Video Resolution, and since the 
Layout Resolution includes a 1:1 Font PAR, the header would need to override 
that...

Original comment by TheDarkS...@googlemail.com on 29 Jun 2013 at 10:04

[GoogleCodeExporter]

>But isn't the \pos tag always stretched/squeezed from Script Resolution to 
Output Resolution without AR concerns anyway?  I can't imagine a situation 
where not stretching it to Output Resolution but instead to the AR-keeping 
Output Resolution would be beneficial...

Yes. I don't know, maybe it would be better for total video independence. 
Anyway, it was just a possibility for what the semantics could be.

>The font stretching you mention is only needed because the Font PAR at the 
video's resolution

Ther are two separate concerns. PAR of the output image, and PAR of the fonts. 
mplayer2/mpv can do both.

For one, it can render unstretched font even if the video output stretches the 
pixel output. This happens when playing anamorphic video with 
-no-ass-vsfilter-aspect-compat and -vo xv. In this case, the player actually 
requests "sqeezing" the font to make it narrower. Then it gets stretched again 
on output.

Second, with -ass-vsfilter-aspect-compat and anamorphic video we actually have 
to stretch the output if we render directly to the screen, like with -vo gl.

Third, with -ass-vsfilter-aspect-compat and anamorphic video and -vo xv, the 
font PAR as rendered by libass is 1:1 again. This is exactly the vsfilter 
scenario.

In my opinion,  video resolution independent script mode should never modify 
the font PAR according to the video PAR, right?

>Again, I think I may be missing something.  Isn't ass_set_storage_size() the 
field that tells libass the resolution of the video?  If so, why do you need to 
disable it if you need the Video Resolution?

No. ass_set_storage_size() is ONLY needed to compute the \blur scale, because 
this tag was implemented incorrectly when it was added to vsfilter. This API 
call can also set the aspect ratio, but it's not really needed.

>Also, what Aspect Hack?  If my guess is right, can't you just ignore the Font 
PAR value specified by the calling application and use the PAR calculated from 
the Video Resolution / Output Resolution ratio instead with this header value?

No, it'd be multiplied. It would ignore ass_set_storage_size() and has to call 
ass_set_aspect_ratio() in some cases, I guess.

The "hack" refers to how the vsfilter aspect ratio behavior is emulated by 
letting the application set a certain PAR.

>Or is my guess from above right and the Font PAR is not specified for the 
Output Resolution but for the Video Resolution?

No. But ass_set_storage_size() can set an aspect if the application doesn't use 
ass_set_aspect_ratio().

>I don't understand.  With Layout:  853x480, the Video Resolution is absolutely 
irrelevant, as the Layout Resolution completely replaces the Video Resolution.  
Just pretend that the Layout Resolution _is_ the Video Resolution...

I don't understand either. libass doesn't depend on the video resolution. 
ass_set_storage_size() was added only recently to fix \blur scale. It's not 
even used by all applications.

>Pretty much, yes.  Though this also aims to take care of e.g. a 720x480 script 
that was used with a 1280x720 video that you now want to use on a 720x480 video 
(or a 1920x1080 video) for example.

Should that just work, even now? Assuming the anamorphic aspect ratio is the 
same, and you don't use \blur tags.

Original comment by nfxjfg@googlemail.com on 29 Jun 2013 at 11:18

[GoogleCodeExporter]

[deleted comment]

[GoogleCodeExporter]

I deleted my prvious comment because I forgot to remove all the newlines and 
such before pasting.  Sorry about that.

> >720x480 script that was used with a 1280x720 video
> Should that just work, even now? Assuming the anamorphic aspect ratio is the 
same, and you don't use \blur tags.
In itself, without re-using the script on another video, this should work.  The 
video's PAR is 1:1, so even standard VSFilter rendering doesn't need 
corrections (it could be a script designed for VSFilter's PAR Compensation 
feature, for example), so this should work fine as long as the text isn't 
squeezed because of some weird AR setting somewhere.  About reusing the script 
on a video of another resolution, well, currently that doesn't work unless you 
manually specify the necessary options (and ass_set_storage_size() gets the 
1280x720 resolution).
Also, aside from \blur and \be, I don't think this has much relevant 
information for libass.
> >\pos tag
> I don't know, maybe it would be better for total video independence.
I understand.  I'll bring it up in the xy-VSFilter issue, but I don't expect 
this behavior to be approved, as I can see it breaking compatibility.  For 
example, the case of 720x480 script on 1280x720 video would break from this... 
(by the way, I just noticed that Issue #6 uses that configuration, 720p video 
with 720x480 script)

> Ther are two separate concerns. PAR of the output image, and PAR of the 
fonts. mplayer2/mpv can do both.
Thank you, I think I have a better understanding of things now.
Looking through Issue #6 again, I'm thinking about how the Layout Resolution 
would actually affect libass rendering...  as far as I can see, it should only 
influence rendering of \blur (and maybe \be), aside from indirectly defining 
font_scale_x at Script Resolution.  I suppose that it could also be used with 
ScaledBorderAndShadow: no scripts, but that isn't working right now anyway from 
what I've heard.
I suppose that font_scale_x, which seems to be the value that all lines are 
going to be stretched by after applying \fscx to them, is set at Script 
Resolution, and not at Output Resolution?  In that case, is it possible to 
simply set it to the correct value from the ratio between Layout and Script 
Resolution?  It should be something like
font_scale_x = MonitorPAR * (LayoutY / PlayResY) * (PlayResX / LayoutX)
at Script Resolution, and from then on, scaling without Aspect Ratio concerns 
should be possible.  I am not sure if Monitor PAR is relevant at this point 
already (I'm not yet sure how to understand the concept), so I included it, and 
it can be removed if it's irrelevant.
> In my opinion,  video resolution independent script mode should never modify 
the font PAR according to the video PAR, right?
I use font_scale_x instead of Font PAR on purpose, because I am not entirely 
sure I have been using the term correctly.
I am not sure if I agree.  The font squeezing from Case 1 can be achieved by 
using Layout: 853x480 (or anything else that's [close to] 16:9) within the 
script, which results in a font_scale_x that is not MonitorPAR for 720x480.  
Cases 2 and 3 can both achieve the result with Layout: 720x480 to achieve the 
same result:  The font_scale_x that should be used can be derived completely 
from Layout and Script Resolution without a need for the Video Resolution.  
However, the Video PAR becomes somewhat related to the font_scale_x value due 
to this...

> ass_set_storage_size() is ONLY needed to compute the \blur scale
Then the ass_set_storage_size() field would still need to be filled by the 
Layout Resolution, because as I mentioned above, libass' use for Layout 
Resolution would mainly be Aspect Ratio and \blur related, so I was wondering 
if when the Layout: Compatible (video resolution) value is used, the 
ass_set_storage_size() couldn't simply be re-routed to set the Aspect Ratio at 
Script Resolution as well.  Is there a way to get the Video Resolution for 
setting the font_scale_x parameter correctly, from ass_set_storage_size() or 
from elsewhere?

> It would ignore ass_set_storage_size() and has to call ass_set_aspect_ratio() 
in some cases, I guess.
This may be.  However, when Layout: PlayRes is used, font_scale_x at PlayRes is 
MonitorPAR.

> I don't understand either. libass doesn't depend on the video resolution.
At that point, I only mentioned Video Resolution being replaced by Layout 
Resolution because your list of confusing resolutions mentioned the video 
resolution...  I guess that confused me in turn, so I'm sorry about that.

> Should that just work, even now? Assuming the anamorphic aspect ratio is the 
same, and you don't use \blur tags.
Assuming I don't use blur tags, it should normally work fine with libass as 
long as I compensate.  However, knowing how to compensate for which scripts is 
part of the reason I wanted a Layout Resolution tagged, so that it can be 
automated.  Also, I believe that in the long term, it will help people to move 
to using Script Resolution as base for everything instead of Video Resolution.

Original comment by TheDarkS...@googlemail.com on 1 Jul 2013 at 8:38

[GoogleCodeExporter]

>I am not sure if Monitor PAR is relevant at this point already (I'm not yet 
sure how to understand the concept),

In the end, the renderer (like the video output) wants to set a specific PAR, 
but this is purely an aspect about the way the end result is displayed. libass 
just needs to honor the PAR, and doesn't have to care about it otherwise.

There are two reasons why you'd need this display PAR:
1. Use on a monitor with a non-1:1 PAR, like TV monitors. mplayer/mplayer2/mpv 
can do this with the -monitorpixelaspect option. This is quite obscure and 
maybe nobody uses it, but it is supported.
2. Displaying anamorphic video with vo_xv. Xv does not support video overlays 
or OSD, so it has to be emulated by drawing into the video directly. This means 
whatever you render into the video, it will be essentially be stretched (like 
the video) on display. So, to get a 1:1 PAR, you have to render your stuff at a 
compensating PAR.

Both of these PARs are multiplied by the application and set via 
ass_set_aspect_ratio()  or ass_set_pixel_aspect(). (Which are the same thing 
really... the former function has sar and dar parameters, but libass just 
multiplies these, and all libass users pass in 1 for one of these parameters.)

>> In my opinion,  video resolution independent script mode should never modify 
the font PAR according to the video PAR, right?
>I use font_scale_x instead of Font PAR on purpose, because I am not entirely 
sure I have been using the term correctly.
>I am not sure if I agree.

Well, it's way too easy to confuse things here and having people talking about 
different things without noticing. What I meant is: the video PAR shouldn't 
influence how you see the fonts on the screen. But this is what classic 
vsfilter does with anamorphic video, so there's explicit compensation for this 
(which you can disable with -no-ass-vsfilter-aspect-compat).

AFAIK older libass versions actually changed the PAR according to PlayRes, but 
this was disabled.

>Is there a way to get the Video Resolution for setting the font_scale_x 
parameter correctly, from ass_set_storage_size() or from elsewhere?

I'm not sure what you mean. ass_set_storage_size() is defined to always use the 
video storage size (for anamorphic DVD video, that would be 720x480). Other 
than that, no, there's no way to know the video anamophic display size. But you 
can get the aspect ration, because you know the video display size. The display 
size is usually scaled, but since that doesn't affect the aspect ratio, you can 
derive the aspect ratio from it.

Original comment by nfxjfg@googlemail.com on 3 Jul 2013 at 9:32

[GoogleCodeExporter]

I'll use "Layout: script | video | WxH" instead of "Layout: PlayRes | 
Compatible | WxH" from now on, because it's shorter and I believe also more 
intuitive.

> There are two reasons why you'd need this display PAR:
> 1. Use on a monitor with a non-1:1 PAR, like TV monitors. 
mplayer/mplayer2/mpv can do this with the -monitorpixelaspect option.
>    This is quite obscure and maybe nobody uses it, but it is supported.
> 2. Displaying anamorphic video with vo_xv. Xv does not support video overlays 
or OSD, so it has to be emulated by drawing into
>    the video directly. This means whatever you render into the video, it will 
be essentially be stretched (like the video)
>    on display. So, to get a 1:1 PAR, you have to render your stuff at a 
compensating PAR.
Thank you for your explanation.  Monitor Pixel Aspect Ratio should be of no 
concern at all to this issue, then, although I think it's better if I explain 
that later.

> the video PAR shouldn't influence how you see the fonts on the screen. But 
this is what classic vsfilter does with
> anamorphic video, so there's explicit compensation for this (which you can 
disable with -no-ass-vsfilter-aspect-compat).
I think I disagree at least partially:  In my opinion, most ASS scripts are 
created for a certain image of a certain AR, and that the subtitles should 
imitate exactly what happened to that image if something is different.  If, for 
example, that certain image of a certain AR has the resolution of 1280x720 
(Layout: 1280x720), then the subtitles should be horizontally stretched when 
they are overlaid onto an image of, say, 1400x720 resolution, along with 
borders etc., without paying special attention to the AR mismatch (simply scale 
in x direction only).  Similarly, the subtitles should be rendered squished 
onto an image of 720x480 resolution, again leaving the AR mismatch as it is.  
Generally speaking, libass would render lines like this:

ass_set_storage_size() = Layout
font_scale_x = (LayoutY / PlayResY) * (PlayResX / LayoutX)
fscx = fscx * font_scale_x * (OutputX / PlayResX) = fscx * ((LayoutY * OutputX) 
/ (PlayResY * LayoutX))
fscy = fscy * (OutputY / PlayResY)

These fscx and fscy values replace the lines' fscx and fscy values.  This makes 
setting the Monitor PAR for libass irrelevant because for -vo vx, the subtitles 
are requested at video resolution, anyways, so the subtitles will be scaled the 
same way the video is (if libass compensated for the Monitor PAR, subtitles 
would look distorted).  With -vo gl, the Output Resolution changes along with 
Monitor PAR (e.g. from a normal 1920x1080 to 1950x1080), so telling anything 
other than the Output Resolution is unnecessary in this case as well.  I 
believe that when, for whatever reason (Monitor PAR, a strange user's 
preference, ...), a video is stretched or squished to have an AR different from 
Layout AR, the subtitles should be stretched or squished accordingly, as if 
they had been created at Layout Resolution and then scaled to the current 
Output Resolution.I think this is important because Typesetting, for example, 
often depends on the video contens being essentially the same, regardless of 
whether they're at 720x480 or 1920x1080 resolution.  If then, the subtitles are 
scaled differently for one of these two versions (it doesn't matter if 720x480, 
1920x1080 or maybe an entirely different resolution is used for Layout,a s long 
as it matches between scripts), this will lead to incorrect display on at least 
one of the video's versions.
If you interpret this as "the video PAR shouldn't influence how you see the 
fonts on the screen", then I agree.  However, when a 16:9 video is scaled to 
4:3 just because of a strange user preference, then the subtitles' look is kind 
of influenced by video PAR again...
Also, I'll be honest, I expect that if this is implemented, more DVD-resolution 
videos will use scripts with Layout: 720x480 (kind of confirming the usage of 
-ass-vsfilter-aspect-compat over -no-ass-vsfilter-compat).  This however won't 
matter much, because the rendering to any resolution is unambiguously defined 
with the Layout flag.  Also, because the flag is optional, libass can simply 
fallback to using the player-specified values for scripts that aren't flagged 
(probably old scripts and scripts without Typesetting in them).

> AFAIK older libass versions actually changed the PAR according to PlayRes, 
but this was disabled.
It sounds like this is expected behavior for the Layout: script scenario.  This 
is also visible in the fscx formula above (adapted for PlayRes = Layout):
fscx = fscx * ((PlayResY * OutputX) / (PlayResY * PlayResX)) = fscx * (OutputX 
/ PlayResX)

> ass_set_storage_size() is defined to always use the video storage size (for 
anamorphic DVD video, that would be 720x480).
> Other than that, no, there's no way to know the video anamophic display size.
I forgot that at the beginning of our discussion (c17), you said that
> Sounds perfect, especially if it's just "Layout: Compatible", and no 
additional resolution or anything is specified.
> Not much would change for libass. It'd have to export the header to the 
application (so that it disable the damn
> aspect hack), and it'd have to ignore ass_set_storage_size().
This means that libass doesn't even need to know the video resolution in this 
case [except for blur scaling with ass_set_storage_size()].  However, thinking 
about it now, I think it would mean that the application should enable the 
aspect hack (-ass-vsfilter-aspect-compat) in that case, I think?  Can you 
confirm this?  And why would it have to ignore ass_set_storage_size()?

Original comment by TheDarkS...@googlemail.com on 5 Jul 2013 at 2:08

[GoogleCodeExporter]

I made a short summary which I hope is sufficient.

What I want is the ability to tag a resolution to the script at which it is 
assumed that the script has been authored at.  Because of that, this resolution 
is used to fill the ass_set_storage_size() field, and also to calculate the 
Font PAR at any Output Resolution like this:
 FontPAR = (OutputX / LayoutX) : (OutputY / LayoutY)
Aside from that, there should be no influence to scaling for libass, and e.g. 
the \pos continues being scaled from PlayRes.

This tag brings the benefit of being able to re-use any (tagged) ASS script 
with any video without the need to adjust e.g. the \blur values to the new 
resolution.  Furthermore, there is no need to guess what Font PAR should be 
requested for a given script with anamorphic video.
The additional resolution which I call Layout Resolution is tagged in the 
[Script Info] section of the ASS file.  The header is completely optional and 
when it is not present, no changes to the current way of subtitle rendering are 
made.  The tag is "Layout: WxH" with two special values, "script" and "video" 
to be used instead of the WxH format.
> AFAIK older libass versions actually changed the PAR according to PlayRes, 
but this was disabled.
For "Layout: script", this behavior can simply be re-enabled, and 
ass_set_storage_size() can be set to PlayRes (or it can be disabled, the result 
should currently be the same).
For "Layout: video", libass can signal to the player to calculate the Font PAR 
it should request the following way:
 FontPAR = (ScaledVideoX / UnscaledVideoX) : (ScaledVideoY / UnscaledVideoY)
For "Layout: WxH", the Font PAR value libass should output subtitles at can be 
calculated as I wrote above.

For example, scaling an ASS script with "Layout: 720x480" to an Output 
Resolution of 1920x1080 should result in a Font PAR of
 FontPAR = (1920 / 720) : (1080 / 480)

Is this something you can agree with?

Original comment by TheDarkS...@googlemail.com on 7 Jul 2013 at 8:08

[GoogleCodeExporter]

I still don't understand what's the purpose. Some kind of elaborate scheme to 
make it compatible with old(/new) vsfilter versions?

Why not just write a script that "resamples" an ASS script if you want to reuse 
it?

Original comment by nfxjfg@googlemail.com on 9 Jul 2013 at 9:59

[GoogleCodeExporter]

> Why not just write a script that "resamples" an ASS script if you want to 
reuse it?
Because this only takes care of the symptom instead of resolving the problem, 
which is the dependence on video resolution.

> I still don't understand what's the purpose.
The purpose is to make rendering of an ASS script at any resolution independent 
from the resolution of the video that the subtitles are rendered for.

> Some kind of elaborate scheme to make it compatible with old(/new) vsfilter 
versions?
About compatibility with old VSFilter versions:  Personally, I don't like the 
"Layout: video" option at all, I'm fine with "script" and "WxH".  However, in 
the xy-VSFilter issue, the libass project members wanted an option to choose 
from either video or script resolution, so I included it in the proposal.
I have always seen this as a way to make subtitles render correctly with new 
subtitle renderers, not as a way to purposefully keep compatibility with old 
VSFilter.

Original comment by TheDarkS...@googlemail.com on 13 Jul 2013 at 2:23