Embedded subtitles does not show on macOS

[Him188]

Code:

val resp = player.subpictures().setTrack(id)
logger.info { "Set subtitle track to $id (${track.labels.firstOrNull()}). resp=$resp" }
logger.info { "All ids: ${player.subpictures().trackDescriptions().map { it.id() }}" }

Logs:

2024-05-21 18:12:55,081 [INFO ] VlcjVideoPlayerState: Set subtitle track to 4 (Label(language=null, value=中文（繁轉簡） - [Chinese])). resp=4
2024-05-21 18:12:55,081 [INFO ] VlcjVideoPlayerState: All ids: [-1, 2, 3, 4]

VLCJ does not show subtitles

MPV shows subtitles correctly

macOS aarch64 vlc version 3.0.20

VLCJ player is from a CallbackMediaPlayerComponent:

    val component = run {
        object : CallbackMediaPlayerComponent() {
            override fun mouseClicked(e: MouseEvent?) {
                super.mouseClicked(e)
                parent.dispatchEvent(e)
            }
        }
    }
    val player: EmbeddedMediaPlayer = component.mediaPlayer()

[Him188]

Looks like the same issue happens to all videos with embedded subtitles, not just this particular one.

[caprica]

What happens when you play your video in VLC?

[caprica]

And also, when are you calling setTrack()? Are you calling it after the media has started (media player "ready" event)?

[Him188]

What happens when you play your video in VLC?

It works well, exactly same like MPV.

And also, when are you calling setTrack()? Are you calling it after the media has started (media player "ready" event)?

Yes, after I can see the video is playing.

[Him188]

Got some logs

[ass] libass API version: 0x1701000
[ass] libass source: tarball: 0.17.1
[ass] Shaper: FriBidi 1.0.12 (SIMPLE) HarfBuzz-ng 3.4.0 (COMPLEX)
[ass] Using font provider coretext
[ass] fontselect: (JGL659OD, 400, 0) -> JGL659OD, 0, JGL659OD
[ass] fontselect: (GJHBRTNT, 400, 0) -> GJHBRTNT, 0, GJHBRTNT

[Him188]

I seek to about 01:00 then seek back to 00:05, then I can see subtitles working. Very interesting.

But this does not always work. Actually I got it work only 2 times of 10

[caprica]

When you invoke the vlcj API to select the SPU, all it does really is to call a corresponding native LibVLC method.

So the most likely issue is that you are calling it too early, before VLC knows about the SPU tracks. The tracks are not known until some short time after the media has started playing.

In your first post, you should move the logger that logs the track descriptions to BEFORE you make the call to change the SPU track, what does it log if you do that?

[Him188]

I turned "-vv" and see error logs repeatedly even if I paused the video:

[00000001212d1880] main blend error: blending RGBA to CVPP failed
[00000001212d1880] main blend debug: looking for video blending module matching "any": 1 candidates
[00000001212d1880] blend blend error: no matching alpha blending routine (chroma: RGBA -> CVPP)
[00000001212d1880] main blend debug: no video blending modules matched
[00000001212d1880] main blend error: blending RGBA to CVPP failed
[00000001212d1880] main blend debug: looking for video blending module matching "any": 1 candidates
[00000001212d1880] blend blend error: no matching alpha blending routine (chroma: RGBA -> CVPP)
[00000001212d1880] main blend debug: no video blending modules matched
[00000001212d1880] main blend error: blending RGBA to CVPP failed
[00000001212d1880] main blend debug: looking for video blending module matching "any": 1 candidates
[00000001212d1880] blend blend error: no matching alpha blending routine (chroma: RGBA -> CVPP)
[00000001212d1880] main blend debug: no video blending modules matched
[00000001212d1880] main blend error: blending RGBA to CVPP failed
[00000001212d1880] main blend debug: looking for video blending module matching "any": 1 candidates
[00000001212d1880] blend blend error: no matching alpha blending routine (chroma: RGBA -> CVPP)
[00000001212d1880] main blend debug: no video blending modules matched
[00000001212d1880] main blend error: blending RGBA to CVPP failed
[00000001212d1880] main blend debug: looking for video blending module matching "any": 1 candidates
[00000001212d1880] blend blend error: no matching alpha blending routine (chroma: RGBA -> CVPP)
[00000001212d1880] main blend debug: no video blending modules matched
[00000001212d1880] main blend error: blending RGBA to CVPP failed
[00000001212d1880] main blend debug: looking for video blending module matching "any": 1 candidates
[00000001212d1880] blend blend error: no matching alpha blending routine (chroma: RGBA -> CVPP)
[00000001212d1880] main blend debug: no video blending modules matched
[00000001212d1880] main blend error: blending RGBA to CVPP failed
[00000001212d1880] main blend debug: looking for video blending module matching "any": 1 candidates
[00000001212d1880] blend blend error: no matching alpha blending routine (chroma: RGBA -> CVPP)
[00000001212d1880] main blend debug: no video blending modules matched
[00000001212d1880] main blend error: blending RGBA to CVPP failed
[00000001212d1880] main blend debug: looking for video blending module matching "any": 1 candidates
[00000001212d1880] blend blend error: no matching alpha blending routine (chroma: RGBA -> CVPP)
[00000001212d1880] main blend debug: no video blending modules matched
[00000001212d1880] main blend error: blending RGBA to CVPP failed
[00000001212d1880] main blend debug: looking for video blending module matching "any": 1 candidates

By selecting -1 track the error log stopped.

[Him188]

I have a UI to select subtitle track so VLC must have already parsed the tracks.

---

Still tried though:

logger.info { "All ids: ${player.subpictures().trackDescriptions().map { it.id() }}" }
player.subpictures().setTrack(id)
logger.info { "Set subtitle track to $id (${track.labels.firstOrNull()})" }

2024-05-21 21:14:04,150 [INFO ] VlcjVideoPlayerState: All ids: [-1, 2, 3]
2024-05-21 21:14:04,150 [INFO ] VlcjVideoPlayerState: Set subtitle track to 2 (Label(language=null, value=chs - [Chinese]))

[caprica]

So that's exactly what I do in the vlcj-player demo application, and subtitles have always worked there. Really not sure what else could be wrong.

[Him188]

Looks like to an issue of vlc itself.

[caprica]

Is the media you're testing publicly available?

When I test my own media with subtitles, using the vlcj-player demo app, I don't see this problem.

[Him188]

I'm afraid it's an issue with VLC on macos. From the logs, it seems VLC is trying to blend the video and the subtitle, which is likely to use some hardware API. I'm on a AArch64 Macbook M2 Max, which has Metal API and VideoToolbox for codec. Some other people also have this issue years ago on Linux. They suggested --avcodec-hw none to disbale hardware acceleration but it did not work for me. I spent 3 hours on this and did not solve it.

I'm going to test it on Windows.

The video:

magnet:?xt=urn:btih:d8e9eacc8c1d8037363f04a309a4ed43abe4d60d&dn=%5BKitaujiSub%5D%20Hibike%21%20Euphonium%203%20%5B07%5D%5BWebRip%5D%5BHEVC_AAC%5D%5BCHS_JP%26CHT_JP%5D.mkv&tr=udp%3A%2F%2Ftr.bangumi.moe%3A6969%2Fannounce&tr=udp%3A%2F%2Foh.fuuuuuck.com%3A6969%2Fannounce&tr=udp%3A%2F%2Fp2p.publictracker.xyz%3A6969%2Fannounce&tr=udp%3A%2F%2Fretracker01-msk-virt.corbina.net%3A80%2Fannounce&tr=udp%3A%2F%2Ftracker.0x7c0.com%3A6969%2Fannounce&tr=udp%3A%2F%2Ftracker.picotorrent.one%3A6969%2Fannounce&tr=udp%3A%2F%2Ftracker.tiny-vps.com%3A6969%2Fannounce&tr=udp%3A%2F%2Ftracker.tryhackx.org%3A6969%2Fannounce&tr=http%3A%2F%2Fbvarf.tracker.sh%3A2086%2Fannounce&tr=http%3A%2F%2Ftracker.gbitt.info%3A80%2Fannounce&tr=https%3A%2F%2Ftracker.gbitt.info%3A443%2Fannounce&tr=udp%3A%2F%2Ftracker-udp.gbitt.info%3A80%2Fannounce&tr=https%3A%2F%2Ftracker.loligirl.cn%3A443%2Fannounce&tr=https%3A%2F%2Ftracker1.ctix.cn%3A443%2Fannounce&tr=udp%3A%2F%2Fmoonburrow.club%3A6969%2Fannounce&tr=http%3A%2F%2Ft.nyaatracker.com%2Fannounce&tr=http%3A%2F%2Fopen.acgtracker.com%3A1096%2Fannounce&tr=http%3A%2F%2Fopen.nyaatorrents.info%3A6544%2Fannounce&tr=http%3A%2F%2Ft2.popgo.org%3A7456%2Fannonce&tr=http%3A%2F%2Fshare.camoe.cn%3A8080%2Fannounce&tr=http%3A%2F%2Fopentracker.acgnx.se%2Fannounce&tr=http%3A%2F%2Ftracker.acgnx.se%2Fannounce&tr=http%3A%2F%2Fnyaa.tracker.wf%3A7777%2Fannounce&tr=udp%3A%2F%2Ftracker.publicbt.com%3A80%2Fannounce&tr=http%3A%2F%2Ftracker.kamigami.org%3A2710%2Fannounce&tr=http%3A%2F%2Fanidex.moe%3A6969%2Fannounce&tr=http%3A%2F%2Ft.acg.rip%3A6699%2Fannounce&tr=https%3A%2F%2Ftr.bangumi.moe%3A9696%2Fannounce&tr=http%3A%2F%2Ftr.bangumi.moe%3A6969%2Fannounce&tr=udp%3A%2F%2Ftracker.opentrackr.org%3A1337%2Fannounce&tr=udp%3A%2F%2Ftracker.torrent.eu.org%3A451%2Fannounce&tr=udp%3A%2F%2Fopentracker.i2p.rocks%3A6969%2Fannounce&tr=udp%3A%2F%2Fopen.stealth.si%3A80%2Fannounce&tr=https%3A%2F%2Fopentracker.i2p.rocks%3A443%2Fannounce&tr=udp%3A%2F%2Ftracker.pomf.se%3A80%2Fannounce&tr=udp%3A%2F%2Ftracker.openbittorrent.com%3A6969%2Fannounce&tr=udp%3A%2F%2Ftracker.moeking.me%3A6969%2Fannounce&tr=udp%3A%2F%2Ftracker.dler.org%3A6969%2Fannounce&tr=udp%3A%2F%2Fp4p.arenabg.com%3A1337%2Fannounce&tr=udp%3A%2F%2Fopen.demonii.com%3A1337%2Fannounce&tr=udp%3A%2F%2Fmovies.zsw.ca%3A6969%2Fannounce&tr=udp%3A%2F%2Fipv4.tracker.harry.lu%3A80%2Fannounce&tr=udp%3A%2F%2Fexplodie.org%3A6969%2Fannounce&tr=udp%3A%2F%2Fexodus.desync.com%3A6969%2Fannounce&tr=udp%3A%2F%2F9.rarbg.com%3A2810%2Fannounce&tr=https%3A%2F%2Ftracker.nanoha.org%3A443%2Fannounce&tr=https%3A%2F%2Ftracker.lilithraws.org%3A443%2Fannounce&tr=https%3A%2F%2Ftr.burnabyhighstar.com%3A443%2Fannounce&tr=http%3A%2F%2Ftracker.openbittorrent.com%3A80%2Fannounce&tr=http%3A%2F%2Ftr.nyacat.pw%2Fannounce&tr=https%3A%2F%2Ftr.nyacat.pw%2Fannounce

[Him188]

I seek to about 01:00 then seek back to 00:05, then I can see subtitles working. Very interesting.

This does not work anymore. Actually, this looks like to be a bug. My app supports streaming and it might accidentally read file chunk that has not yet been downloaded, causing VLC to fail and it might switch to some fallback codec/blending/rendering strategies, so it worked.

But now when the file is fully downloaded, seeking does not cause problem and I'm not able to make it work with this seeking trick.

However this gives a clue that the subtitles can work on my machine but need some tricks.

[Him188]

The exactly same code works well on Windows.

[caprica]

This really sounds like it's not a vlcj issue then.

[Him188]

This really sounds like it's not a vlcj issue then.

Does vlcj adds any vlc flags? what are they?

[Him188]

Some people suggest that vlc 4.0 works. I might give it a try in later days. For now I would focus more on Windows since it looks like to have better support and most of my desktop users are on Windows. Bought you an albulm BTW, thanks for the help.

Should we keep this issue on and I can share workarounds in the future?

[caprica]

Thank you very much, that's very kind!

By all means feel free to post any updates here.

[caprica]

btw what do you mean by "flags"?

In this case vlcj simply calls libvlc_set_subtitle_track(mediaplayer, id) - or whatever the API is called.

[Him188]

btw what do you mean by "flags"?

command line flags to start vlc. like -vv to enable very verbose logs.

[caprica]

Then no, vlcj does not pass any of those by default. You can pass them when you create the MediaPlayerFactory though.

[Him188]

Is it because the Callback approach uses RV24 which is RGB? This matches the log that vlc cannot blend RGBA with CVPP

Is it possible for vlcj to use other formats like YUV?

[caprica]

IMO that is unlikely.

Nevertheless, you are free to use whatever buffer format you want... however... you have to first pick something supported by VLC of course, but then you have pick something that can be rendered by your video surface. If you're using the CallbackMediaPlayerComponent you can provide your own BufferFormat instance.

For BufferedImage, this is basically RGBA (hence RV32 *). If you have a different renderer, or can set a different colour model or something, then you experiment with the other formats.

The reason I put "*" there is to note that with VLC 4.x and/or latest versions of JDKs, something changed with the buffer formats which meant RV32 no longer rendered correctly and instead BGRA is used. I don't quite understand it tbh, it seems the wrong format to me but I'm no expert on chroma models.

It's been a little while since I tested latest vlcj with latest VLC, but certainly recently this was the case.

[caprica]

YUV is unlikely as far as I know since nothing in Java can render YUV can it?

[Him188]

I'm not an expert either, but would take your advice. Thanks. I might try other formats later.

[Him188]

I tried the YUV format I422, but VLC is still trying to blend RGBA.

[000000035bee1010] blend blend error: no matching alpha blending routine (chroma: RGBA -> CVPP)
[000000035bee1010] main blend error: blending RGBA to CVPP failed