image-based subtitles are stretched to video aspect ratio

[visibleonlynow]

mpv Information

mpv v0.38.0-dirty Copyright © 2000-2024 mpv/MPlayer/mplayer2 projects
 built on May 23 2024 10:29:03
libplacebo version: v6.338.2
FFmpeg version: n6.1.1
FFmpeg library versions:
   libavutil       58.29.100
   libavcodec      60.31.102
   libavformat     60.16.100
   libswscale      7.5.100
   libavfilter     9.12.100
   libswresample   4.12.100

Important Information

• Platform version: Linux 6.9.2-zen1-1-zen
• GPU model, driver and version: NVIDIA GeForce RTX 2080, nvidia-dkms 550.78
• Source: https://archlinux.org/packages/extra/x86_64/mpv/ Version 1:0.38.0-4
• Introduced in version: ?

screenshots

using video-aspect-override=16:9

Reproduction Steps

To reproduce this, simply play any non-16:9 video with image-based subtitles (in my case dvd_subtitle) and they will appear stretched. This also happens in VLC and MPC-HC, but not in Infuse. When using video-aspect-override=16:9 the subtitles are displayed correctly.

Expected Behavior

Image-based subtitles should be displayed in 16:9, even when the video is a different aspect ratio.

This is what the same file looks like in Infuse:

Actual Behavior

The subtitles are stretched to the same aspect ratio as the video. See the screenshots in Important Information

Log File

output.txt

Sample Files

No response

I carefully read all instruction and confirm that I did the following:

• [X] I tested with the latest mpv version to validate that the issue is not already fixed.
• [X] I provided all required information including system and mpv version.
• [X] I produced the log file with the exact same set of files, parameters, and conditions used in "Reproduction Steps", with the addition of --log-file=output.txt.
• [X] I produced the log file while the behaviors described in "Actual Behavior" were actively observed.
• [X] I attached the full, untruncated log file.
• [X] I attached the backtrace in the case of a crash.

[ruihe774]

You're using a VOBsub subtitle. Please extract it and attach it here (both .idx and .sub). Stretching can happen if size in VOBsub does not match the video. Are they ripped from the same disk?

[visibleonlynow]

I'm not sure about the legality of uploading the whole .sub file here, but this is the header of the .idx:

# VobSub index file, v7 (do not modify this line!)
size: 1920x1080
org: 0, 0
scale: 100%, 100%
alpha: 100%
smooth: OFF
fadein/out: 0, 0
align: OFF at LEFT TOP
time offset: 0
forced subs: OFF
palette: 000000, f0f0f0, cccccc, 999999, 3333fa, 1111bb, fa3333, bb1111, 33fa33, 11bb11, fafa33, bbbb11, fa33fa, bb11bb, 33fafa, 11bbbb
custom colors: OFF, tridx: 1000, colors: 000000, 444444, 888888, cccccc
langidx: 0

The size is indeed different than the video size, which is 3840x1600. I don't think this is an issue with this file specifically, as I have experienced the same behaviour with many other files. However, as mentioned in the issue, these exact subtitles are displayed correctly in Infuse, so maybe there is a way to make mpv behave in the same way?

[kasper93]

This also happens in MPC-HC

Works fine, in older version. There was logic to fit uncropped subtitles onto cropped video.

Image-based subtitles should be displayed in 16:9, even when the video is a different aspect ratio.

This is arguable, because we don't know how much of the video were cropped, we can assume that video should be fitted into subtitle size, but in fairness encoder should crop subtitles the same way as video.

I can add an option for this, I find it useful, there are a lot of encodes that have broken subtitles, because people doesn't care.

[visibleonlynow]

I changed the size in the extracted subtitle.idx file to 3840x1600, which matches the video track, and merged it back into the source file. Now the subtitles aren't stretched, but appear somwhere in the middle of the video and are bit smaller, so I doubt that size is the issue, but maybe I just did something wrong.

[kasper93]

Set them to 1920x800

[visibleonlynow]

They look exactly the same with 1920x800.

[cubicibo]

Image subtitles do not work like this. The container only declares the target video dimensions. Changing the width and height fields in the container will not change the coordinates fed to the decoder, nor the transported bitmap data.

Anyway, VobSub should only carry valid DVD resolution subtitles. Why would you ever use this format for HD?

[visibleonlynow]

VobSub should only carry valid DVD resolution subtitles

I don't see how this is going to help me solve the issue. Does this mean that stretching the subtitles is the intended behaviour?

I just found these two options in the man page:

       --stretch-image-subs-to-screen=<yes|no>
              Stretch DVD and other image subtitles to the screen, ignoring
              the video margins. This has a similar effect as
              --sub-use-margins for text subtitles, except that the text
              itself will be stretched, not only just repositioned. (At least
              in general it is unavoidable, as an image bitmap can in theory
              consist of a single bitmap covering the whole screen, and the
              player won't know where exactly the text parts are located.)

              This option does not display subtitles correctly. Use with care.

              Disabled by default.

       --image-subs-video-resolution=<yes|no>
              Override the image subtitle resolution with the video resolution
              (default: no). Normally, the subtitle canvas is fit into the
              video canvas (e.g. letterboxed). Setting this option uses the
              video size as subtitle canvas size. Can be useful to test broken
              subtitles, which often happen when the video was trancoded,
              while attempting to keep the old subtitles.

However, using --stretch-image-subs-to-screen=yes makes the subtitles appear in the middle of the video again. The other flag doesn't help either, but at least there is a mention of image subtitles and stretched in the man page.

Maybe a flag like --image-subs-aspect-override=16:9 could be added to force image-based subtitles to be displayed like they are when using --video-aspect-override=16:9 without affecting the video aspect ratio?

[cubicibo]

I don't see how this is going to help me solve the issue.

Image subtitles (VobSub, DVB-Bitmap, PGS, TTML-PNG) are always generated for a given target video, out of a handful of possible standardized video formats. They can hardly be mixed with arbitrary videos that differs from the one they were supposed to be used on, or files that do not respect the constrained video format.

Does this mean that stretching the subtitles is the intended behaviour?

Not necessarily! The problem is elsewhere: most VobSub subtitles consist of full-screen, and mostly transparent, bitmaps overlaid onto the video. With non-standard video formats, mpv does not know how to manage these gigantic overlays onto a mismatched video format. The only solution is to stretch them. Image subtitles that use proper screen regions (rectangles) are generally correctly handled by mpv.

Hence, the option suggested by @/kasper93 is needed for these "bad" files.

[visibleonlynow]

VobSub subtitles consist of full-screen, and mostly transparent, bitmaps overlaid onto the video

I see, for some reason I just assumed that all image subtitles would be as small as they could be (i.e. from the top left corner to the bottom right of the subtitle). If they are a transparent 1920x1080 image, it makes sense that they appear stretched. I guess this also explains why they appear in the middle of the video when changing something.

Thank you for the explanation; I didn't know about this when creating the issue. Adding a flag as suggested previously would solve this for me.

[crisalis2]

This also happens when using VobSubs from DVDs and videos with different aspect ratios than the DVD's.

One combination I use sometimes is 1920x1080 video and VobSubs (all of them from NTSC or PAL DVD movies that are later displayed at 16:9 including black bars - I think for 4:3 TV shows what I'm about to say would be different). NTSC (720x480) VobSubs are slightly horizontally stretched when used with a 1920x1080 video, and PAL (720x576) ones even more so. Changing the resolution in the .idx file (to 853x480 for NTSC or 1024x576 for PAL) makes the VobSubs display correctly (not stretched), but they are misaligned (not centered, they appear slightly to the left). There are other related parameters you can change in the .idx file, but mpv ignores them (unlike the resolution (size) parameter).

I found that MPC (HC/BE) does apply other .idx parameters, and that the right parameters fix everything. For the record, VLC behaves the same as mpv. For NTSC VobSubs (from movies that are later displayed at 16:9 including black bars) and 1920x1080 video, size: 853x480 org: 67, 0 align: ON at LEFT TOP fixes everything. 67 comes from (853-720)/2. Also note the ON in "align" (default is OFF). For PAL, what fixes it is (again, 154 comes from (1024-720)/2): size: 1024x576 org: 154, 0 align: ON at LEFT TOP

So mpv could honor these .idx parameters, as MPC does.

Another thing that comes to mind is that mpv could let you apply sub-scale separately for x and y, letting you stretch or unstretch subtitles horizontally or vertically however you want (including image-based subtitles). I found https://github.com/mpv-player/mpv/issues/6417 and https://github.com/mpv-player/mpv/commit/0cd8ba72fe47d11b196adc2ad1f64b1243312d2e while searching about all this, and judging by the commit it seems it would be easy to implement? (I don't really know). By the way, the description of --sub-scale in the manual still says text subtitles only, even though it works with image subs since 2019.

[visibleonlynow]

Another thing that comes to mind is that mpv could let you apply sub-scale separately for x and y, letting you stretch or unstretch subtitles horizontally or vertically however you want (including image-based subtitles). I found #6417 and 0cd8ba7 while searching about all this, and judging by the commit it seems it would be easy to implement

Thank you! I hadn't found that issue and commit before. It really does seem trivial to add a sub-scale-x, and a sub-scale-y option. I came up with this patch that adds image-subs-scale-x and image-subs-scale-y. It also decouples sub-scale into sub-scale for regular subtitles and image-subs-scale for image-based subtitles.

(uploaded here as .txt as Github doesn't allow .patch, and it might be too big for a comment) image-subs-scale.patch.txt

Using it with a conditional profile like this:

[scale-image-subs-3840x1600]
profile-desc=Scale DVD subtitles for 3840x1600
profile-cond=(p["current-tracks/sub"]["codec"] == "dvd_subtitle" and p["video-params/w"]==3840 and p["video-params/h"]==1600)
profile-restore=copy
image-subs-scale-x=.73

mitigates the issue.

Still, having to manually do this for different resolutions doesn't seem like a good solution. I think adding something like --image-subs-aspect-override=16:9, as mentioned before, would make more sense, but I couldn't find why (or where) mpv changes image-based subtitles when using video-aspect-override.

[crisalis2]

Still, having to manually do this for different resolutions doesn't seem like a good solution.

If you add keyboard shortcuts to increase/decrease the x/y scaling (maybe only x is necessary), I think it's good enough, you see the result on the fly so adjusting to your liking is easy. Similar to changing the vertical position of the subs on the fly, or other things like adjusting sub/audio delay.