Unifying rendering across resolutions (xy-VSFilter scale function discussion and feedback)

[GoogleCodeExporter]

Okay, so this is somewhat related to the VSFilterForMadVR, but I figured this 
would be better as a separate issue. Basically, if our goal is to render 
subtitles at native desktop resolution (like ISR does), then we need to do some 
changes to certain features like 3D rotation so that no typesetting gets broken 
from the resolution differences. I gave some thought to this and came up with 
the following:

Basically, xy-VSFilter needs to take the script resolution more into account. 
All 3D rotation calculations are made at script resolution, and then "scaled" 
to the rendering resolution. This should be done for both the normal YCbCr 
rendering as well as RGB rendering over the new interface. So basically it 
would mean the following:

1. 3D rotation would still be different at different script resolutions 
(whereas now they are different as long as the rendering resolution is 
different)
2. 3D rotations would be identical across different rendering resolutions as 
long as the script resolution is the same
3. This would mean that if you have a 1280x720 script typeset to 1280x720 video 
with 3D rotations, you could re-use the same script for 1080p, 720p and 480p 
encodes of the same content and the rotations would look identical everywhere 
(this is not currently the case)

With this kind of approach, we would obtain the highest level of backwards 
compatibility - all old scripts that were typeset with script resolution 
identical to the video resolution will still work correctly. What will be 
broken is scripts that were typeset with a script resolution that doesn't match 
the video resolution and involve 3D rotations, but we should just consider 
these scripts to be faulty and ignore them. The only way to be 
backwards-compatible with even this scenario would be pretty much to introduce 
yet another ASS header, and would we really want that?

Beyond 3D rotation, there also other things that should be taken into account 
when rendering at a different resolution from video resolution. Take anamorphic 
video, for instance. If a script has been typeset to 720x480 and for example 
the dialogue style has X-Width of 84% to compensate for the stretching (so that 
it will look normal-width on playback at video resolution rendering), currently 
with ISR it gets rendered incorrectly with 84% width. In other words, 
xy-vsfilter should probably do aspect ratio compensation by default, at least 
with ASS subtitles.

Going back to typesetting, one more thing that comes to my mind is the \be and 
\blur tags - most typesetters use a varying amount of blur for signs in order 
to match them up accurately with the video, so if you suddenly have the script 
rendered at a much higher resolution, these same blur values won't match up 
anymore. As such, in my opinion \be and \blur should also be scaled according 
to script resolution and rendering resolution. Demonstration here: 
http://screenshotcomparison.com/comparison/124672

I don't think this would need anything more complex than simple multiplication 
based on the resolution difference, so if for example a 1280x720 is rendered at 
1920x1080, all the blur values get multiplied by 1.5 (1080/720). Here's what it 
would look like for the comparison linked above: 
http://screenshotcomparison.com/comparison/124673

As you can see, it may not be a "perfect" match for all the elements, but on 
the other hand different scaling algorithms can cause difference in end-product 
sharpness (the comparisons above use Spline 4-tap for both luma & chroma), so I 
think we should just use a simple approach and leave it at that - it will still 
look a lot better than not compensating at all.

Anyway, I guess that's about it for what comes to my mind about this. Thoughts?

Original issue reported on code.google.com by Daiz...@gmail.com on 13 May 2012 at 4:12

[GoogleCodeExporter]

Just to clarify what I meant by ulterior motives, it's not that I necessarily 
believe that Libass has any such motives at the moment, but that I personally 
would have such motives _myself_ if it ever came time to introduce such a tag, 
but I also firmly believe its not the time or the place for such motives 
currently. I find it hard to convince myself that there aren't other people who 
wouldn't use such a boolean tag for the purpose of reform, even if it was 
defined as restricted. At the same time, it wouldn't make much sense to define 
something as restricted if it was going to be re-defined for completely 
different purpose a few years from now. The short-term view of enabling script 
portability is a nice goal, but I'm more concerned when looking at big picture 
and long-term risk assessment. How would people view and use such a header a 
decade from now. As of yet, there are no signs that the ASS format is going to 
be replaced anytime soon (or at all), so this needs to be considered. Views and 
opinions of the majority are subject-to-change over time. Among the many forms 
a new header could take, "do nothing" continues to remain one of the valid 
options.

__________

I'm now going to exit this debate for awhile, since I don't really have 
anything new to add which hasn't already been said, and these endless back and 
forth replies are becoming a bit circular and slightly off-topic (my fault) 
from the core issue at hand at this point. There is really no particular reason 
to rush a decision at the moment. We aren't really in the position to release 
any new build for at least another month or so, and any functionality like this 
would likely be one of the last thing implemented. This will give me time to 
time to think about what's already been discussed, as well as open the door for 
others to enter this discussions to state there opinions or suggestions before 
any final decision is made.

Original comment by cyber.sp...@gmail.com on 21 May 2013 at 1:47

[GoogleCodeExporter]

I'll just say:
Thanks. I think the explanation makes sense, and while I may want to argue with 
it, I'll leave that for later. It does seem we will all benefit from a break.

Original comment by chortos@inbox.lv on 21 May 2013 at 1:59

[GoogleCodeExporter]

Would combining both ideas just make things worse? Something like:

Layout: Compatible | Layout: PlayRes | Layout: WxH

Default to Boolean-like behavior with a string field, but also accept an 
override for the purpose of fixing old scripts?

Original comment by cyber.sp...@gmail.com on 31 May 2013 at 7:17

[GoogleCodeExporter]

I think this is a good idea.
Though, for an easier understanding (and because it's less effort to manually 
type), I propose that the value "Compatible" be called "video" and the value 
"PlayRes" be called "script".  This also has a minor advantage in terminology 
in my opinion, because this way, both value names say where the relevant 
resolution should come from (video or script or specified) while "Compatible" 
and "PlayRes" may be a bit confusing at first (this may just be me, though).

Original comment by TheDarkS...@gmail.com on 2 Jun 2013 at 8:51

[GoogleCodeExporter]

The question of \pos handling at Aspect Ratio mismatches between Script and 
Output Resolution came up in the libass issue.  Currently, the \pos values are 
scaled without caring about AR at all, but the idea came up to scale \pos 
values only as long as the AR is preserved?  This would mean that e.g. 720x480 
Script Resolution would scale its coordinates with 1080p video not to the full 
1920x1080 resolution, but instead only to 1080x1620 resolution...
Currently, nothing does this, but would it make sense at all?  Currently, I can 
only see it breaking e.g. the above-mentioned case, but I'm not sure that's all 
there is to this idea...

Original comment by TheDarkS...@gmail.com on 5 Jul 2013 at 2:06

[GoogleCodeExporter]

So, no answer means I didn't miss anything about the \pos scaling thing?  If I 
get no answer to this question either, I'll just assume so...

Original comment by TheDarkS...@gmail.com on 13 Jul 2013 at 2:17

[GoogleCodeExporter]

Hi, after some hours searching I haven't found a solution.
Please, could anybody point me to information on the following subjets:

1) How can I have bigger subtitles in avisynth?

1.1) Is the xy-VSFilter dll supposed to be loaded as a plugin or C plugin in 
Avisynth (which versions of xy-VSFilter and Avisynth)?

1.2) Is there any doc with an example script showing how to load the needed dll 
and how to call the function with appropriate parameters (or how to setup some 
global default)?

Thanks for any help.

Original comment by NBB.mus...@gmail.com on 31 Dec 2013 at 6:15

[GoogleCodeExporter]

Anybody out there?

I read that quite *some* developers have been putting a great effort into 
faster and better rendering of the subtitles.

Now couldn't we just pass one parameter, a real number, and the font size would 
be the default size multiplied by that number and rounded to an integer?

Am I asking for something too complicated to implement?

Example Avisynth script (where "relFontSize" stays for "relative font size"):

LoadPlugin("<part of path>\AviSynth 2.5\FILTERS\plugins\VSFilter.dll")
LoadCPlugin("<part of path>\AviSynth 2.5\FILTERS\ffms2.dll")
Import("<part of path>\AviSynth 2.5\FILTERS\FFMS2.avsi")

FFmpegSource2("FileName.mp4", vtrack = -1, atrack = -2)
LanczosResize(680, 382)
# or Spline36Resize(680, 382)

textSub("FileName.srt", relFontSize=1.6)

#info()

Original comment by nicola.b...@gmail.com on 20 Jan 2014 at 1:20

[GoogleCodeExporter]

Going back on topic. We discussed this once more in #libass, and it seems we're 
happy with LayoutResX/LayoutResY (original video width/height)!

Original comment by chortos@inbox.lv on 26 Jan 2014 at 4:26

[GoogleCodeExporter]

So it's been almost two years since the creation of this issue, and from 
skimming the thread I guess there's been some sort of conclusion about 
introducing the LayoutResX/Y headers.

If I've understood correctly, and I really hope I have, these new tags would 
allow me to have "the single master script" I talked about in the original post 
- so if I had a single script typeset to 1280x720 video and had its 
ScriptResX/Y and LayoutResX/Y set to 1280x720, I could mux this file with 480p, 
720p and 1080p versions of the same video and get identical (as in correctly 
scaled rendering results) output with all of these files.

Making this possible as soon as possible would be *extremely valuable* due to 
the way 3D rotations work in ASS - trying to scale scripts that use \frx and 
\fry correctly is basically impossible on the authoring end unless you manually 
check and adjust every single instance.

Original comment by Daiz...@gmail.com on 1 Apr 2014 at 7:11

[GoogleCodeExporter]

In terms of xy-VSFilter/XySubFilter, it would be equivalent to using a script 
header to programmatically set: "Renderer Layout Options -> Customize" with 
everything that entails.

Original comment by cyber.sp...@gmail.com on 1 Apr 2014 at 9:13

[GoogleCodeExporter]

Trying it out with Renderer Layout Options in XySubFilter seems to do exactly 
what I want. Any ETA for this feature? It would have immense real-world 
benefits right at this very moment eg. for BD content released in both 1080p 
and 720p. Right now you either need to put in huge effort to do it right or 
release with broken scripts (or avoid \frx and \fry completely), whereas with 
LayoutResX/Y support everything would Just Work(tm).

Also, it would be vital to have this work with regular xy-vsfilter too as long 
as XySubFilter is not feasible to use everywhere - right now changing the 
renderer layout resolution with it seems to have no effect (testing with a 
1280x720 script on a 1920x1080 video).

Original comment by Daiz...@gmail.com on 2 Apr 2014 at 8:06

[GoogleCodeExporter]

I made a couple sample files for testing things with. Here we have 480p, 720p 
and 1080p versions of the same video muxed with a script that has ScriptResX/Y 
and LayoutResX/Y set to 1280x720. The videos have a hardsubbed \frx\fry line on 
bottom and a softsubbed \frx\fry line on top - the lines should look the same 
on all videos with proper LayoutResX/Y support (and you can test it with 
XySubFilter by using "Renderer Layout Options -> Customize"). Libass is already 
on it: https://github.com/libass/libass/pull/72

Original comment by Daiz...@gmail.com on 2 Apr 2014 at 10:48

Attachments:

• layoutres-720p-on-480p.mkv
• layoutres-720p-on-720p.mkv
• layoutres-720p-on-1080p.mkv

[GoogleCodeExporter]

> Any ETA for this feature?

There is no specific ETA, other than it being planned, and our intention to 
sync it with the release of an updated xy-VSFilter build. Since we only have a 
single developer, when he becomes busy with other things development 
effectively stalls. The following is a general list of priorities which we 
would like to complete before making another release:

1) Fix bugs in XySubFilter Beta2
2) Fix bugs which broke xy-VSFilter in latest branches
3) Fix bugs in features which were backed out prior to release of XySubFilter 
Beta2
4) Add support in xy-VSFilter for features added in XySubFilter like RGB color 
correction and Addition Draw
5) Add support for LayoutX/Y header
6) Performance Optimizations(?)
7) ???
8) Release

> normal xy-VSFilter
> right now changing the renderer layout resolution
> with it seems to have no effect

Test again? Setting 'Renderer Layout Output -> Customize -> 1280x720' manually 
has the same effect in both xy-VSFilter and XySubFilter with your samples.

> ScriptResX/Y and LayoutResX/Y set to 1280x720

FWIW, there is no strict requirement that PlayResX/Y needs to match 
LayoutResX/Y for this to function. Any script which is typeset for correct 
output with VSFilter at a particular LayoutResX/Y can be scaled. If for 
example, you typeset all your 3D transforms with the default PlayResX/Y of 
640x480 on a 1280x720 video (LayoutResX/Y) and said script displays correctly 
in VSFilter on a 1280x720 video, it can still be scaled properly. Nowadays you 
don't see this happen much, except on maybe anamorphic video.

Original comment by cyber.sp...@gmail.com on 3 Apr 2014 at 12:01

[GoogleCodeExporter]

> Any ETA for this feature?
> 
> Since we only have a single developer, 
> when he becomes busy with other things development effectively stalls.

If someone contributes a patch to support a LayoutResX/Y header + GUI options 
sometime soon, I'd be willing to release new builds supporting the LayoutResX/Y 
header right now. This would likely mean making an out-of-cycle 'official' 
xy-VSFilter release based on CCCP's branch, along with a new XySubFilter Beta2 
build with the temporary fix for Issue #181.

Original comment by cyber.sp...@gmail.com on 3 Apr 2014 at 2:12