search

jellyfin / jellyfin-web

Web Client for Jellyfin
https://jellyfin.org
GNU General Public License v2.0
2.18k stars 1.16k forks source link

[Issue]: Sub title desync JF 10.8.9 #4346

Open MrToast99 opened 1 year ago

MrToast99 commented 1 year ago

Please describe your bug

Upgraded 10.8.8 > 10.8.9 and now subtitles desync or loop if you jump ahead in a file.

Steps: Start a show with subs, Jump ahead a few mins

This will cause the subs to loop the last 2 or 3 lines constantly.

Turning off subs and back on will resume the subs but they will be desync'd by 2-3 secs. This persists even if playback is stopped and started again.

Jellyfin Version

10.8.0

if other:

No response

Environment

- OS: Windows server 2016
- Virtualization: 
- Clients: browser
- Browser: Chrome
- FFmpeg Version:5.1.2
- Playback Method:
- Hardware Acceleration: no
- Plugins: playback reporting, reports
- Reverse Proxy: yes
- Base URL: /jellyfin
- Networking: NAT
- Storage:NFS

Jellyfin logs

No response

FFmpeg logs

No response

Please attach any browser or client logs here

No response

Please attach any screenshots here

No response

Code of Conduct

poulpor commented 1 year ago

I think I might have the same problem, or somewhat related but I thought it was because of intro-skipper plugin so I opened an issue on ConfusedPolarBear's project. On my setup, only ASS subs are subject to this behavior, it restarts from the beginning of the subtitle file after the intro is skipped. 10.8.8 was doing fine. If you can, could you try the new AndroidTV client 0.15.3, changing the player to libvnc and check if the problem persists?

soultaco83 commented 1 year ago

I am seeing this issue as well on 10.8.9. The subtitles are getting out of sync. Turning them off and on again will fix for a few minutes but then proceeds to go out of sync again. I have reverted to 10.8.8 for the time being. Subtitles are working again without issue. I was using Chrome and Edge to test with.

dmitrylyzo commented 1 year ago
  1. What is the subtitle format (ASS/SRT/PGS)?
  2. Are the subtitles forced to be burnt-in (encoded in video)?
  3. How is the video played (DirectPlay/Remux/DirectStream/Transcode)? FFMpeg logs can also be useful (if it is not DirectPlay).
  4. Is subtitles offset used?
MrToast99 commented 1 year ago

  1. What is the subtitle format (ASS/SRT/PGS)?

    1. Are the subtitles forced to be burnt-in (encoded in video)?

    2. How is the video played (DirectPlay/Remux/DirectStream/Transcode)? FFMpeg logs can also be useful (if it is not DirectPlay).

    3. Is subtitles offset used?

  1. impacts ASS, SRT seems fine
  2. Yes
  3. remux
  4. no FFmpeg.Remux-2023-01-28_08-44-14_45314a29122bc7db3fd72ceabd9e2390_bf4c71d5.log
dmitrylyzo commented 1 year ago

I guess Burn subtitles is set to Auto. Otherwise it would be Transcoding, not Remux. In this case, ASS subs are rendered by JSO (client-side). Since we are using the "renderAhead" mode (cache with timestamps), the error could be due to incorrect timestamps when remuxing (ffmpeg) or possibly a problem with hls.js.

~According to the log, you are using fMP4. Does desync occur when using TS (disable fMP4)?~ This is "Progressive MP4".

UPD: It doesn't seem to work with "Progressive MP4". To reproduce, I had to disable HLS in the web source - don't know how to correctly force progressive mp4. Each time you fast-forward/rewind/seek, subs starts from the beginning because the player is reporting the wrong timestamp. It probably started after https://github.com/jellyfin/jellyfin-web/pull/4182

MrToast99 commented 1 year ago

After looking into your mention of HLS and finding to "Codec Support" article I see that Chrome is marked as no MKV HLS support, where Edge is supported. I tested by continuing played back of a file and jumped around the timeline, the subs loaded perfectly fine.

I did note the sub desync on android a few months back (could correlate to 4182) but I don't playback on android to frequently.

dmitrylyzo commented 1 year ago

After looking into your mention of HLS and finding to "Codec Support" article I see that Chrome is marked as no MKV HLS support, where Edge is supported. I tested by continuing played back of a file and jumped around the timeline, the subs loaded perfectly fine.

Chrome doesn't support the MKV container (the HLS protocol is irrelevant here) and must be remuxed to a supported container. In most cases, it is TS which is streamed via HLS (HLS-TS). You can also enable HLS-fMP4, where the container is MP4, but it will still be transmitted as fragments. Btw, the subtitles seem to work fine with HLS-TS and HLS-fMP4.

In Edge MKV plays as DirectPlay (over HTTP(S) protocol) with correct timestamps.

I did note the sub desync on android a few months back (could correlate to 4182) but I don't playback on android to frequently.

In the Android app, you can enable different players, so there are more variables.

The current problem occurs only when "Progressive MP4" comes into play. This is normal MP4 over HTTP(S) protocol. It was enabled in 10.8.9 (in https://github.com/jellyfin/jellyfin-web/pull/4182) when WebM was replaced with MP4, so StreamBuilder can now use this profile when remuxing or transcoding is needed. I don't know how to get it to be selected, but that's what you have in your log.

What's really going on When you seek, the server starts a new transcoding job - it literally cuts the part from the beginning. In the client (Chrome in your case), the timestamps start at zero (because they have not been saved). There is an offset (seeking position), but it doesn't apply, and the JSO gets them wrong. The reason why it doesn't apply is that the duration of the cut file starts from a small value, and this condition is then incorrect.

To force progressive MP4, I added options.enableHls = false; before this line.

During testing I noticed that ffmpeg doesn't preserve the requested time, so we can't just add an offset - we need to know the actual offset. I was suggested to use -copyts (to ffmpeg command) and that seemed to work, but in this case we don't need to add an offset, and I'm not sure if it doesn't harm other cases.

poulpor commented 1 year ago

Thank you for your researches and the explanation, I suppose it goes the same when, using a web browser like Chrome or Edge and the file is remuxed, the extension intro-skipper does its job and seeks few seconds/minutes ahead: the subtitle then restarts from the beginning. I don't know what web player is used in the Jellyfin Android client, but it acts the same. Exoplayer doesn't play the file and external player works fine but not intro-skipper, which becomes inoperative.

obviously, the problem is not present when the video with ASS sub is directplay, for example by Jellyfin Mac client (which rely on MPV video player), the subs are perfectly synced after intro skipped by the plugin or seeking back and forth.

I hope there will be an easy fix.

cbeamond commented 1 year ago

We've been looking into this at https://github.com/jellyfin/jellyfin/issues/8403. This issue here has some small differences, but it may help figure out some of the problems with the information found in that thread.

MrToast99 commented 1 year ago

Updated to 10.8.10 and this longer seems to an issue...

I'm guessing due to jellyfin/jellyfin-web#4362 or jellyfin/jellyfin-web#4385

dmitrylyzo commented 1 year ago

I'm guessing due to jellyfin/jellyfin-web#4362 or jellyfin/jellyfin-web#4385

https://github.com/jellyfin/jellyfin-web/pull/4385

jellyfin-bot commented 11 months ago

This issue has gone 120 days without comment. To avoid abandoned issues, it will be closed in 21 days if there are no new comments.

If you're the original submitter of this issue, please comment confirming if this issue still affects you in the latest release or master branch, or close the issue if it has been fixed. If you're another user also affected by this bug, please comment confirming so. Either action will remove the stale label.

This bot exists to prevent issues from becoming stale and forgotten. Jellyfin is always moving forward, and bugs are often fixed as side effects of other changes. We therefore ask that bug report authors remain vigilant about their issues to ensure they are closed if fixed, or re-confirmed - perhaps with fresh logs or reproduction examples - regularly. If you have any questions you can reach us on Matrix or Social Media.

AIndoria commented 10 months ago

This remains an issue. Not stale. Subs get desynced when seeking, turning them off and on has no impact. I usually have to force stop and clear cache for it to work again. Incredibly annoying on android/android-tv.

dmitrylyzo commented 10 months ago

This remains an issue. Not stale. Subs get desynced when seeking, turning them off and on has no impact. I usually have to force stop and clear cache for it to work again. Incredibly annoying on android/android-tv.

The problem was originally caused by progressive remuxing, which was dropped in web 10.8.10. But Android probably still has a progressive profile. :thinking: AndroidTV doesn't use web, so that's a different story.

Please answer same questions: https://github.com/jellyfin/jellyfin-web/issues/4346#issuecomment-1426918112 + What player do you use on Android?

AIndoria commented 10 months ago

Please answer same questions: #4346 (comment)

Android Says Web Player. On Android TV Both Exo and libVLC have the same issue (libVLC to a ...lesser extent but it is there). Forcing the video to open in an external player such as MX tends to solve the issue almost completely at times.

ASS plays just fine. SRT/PGS have issues, and it's not all of them. Some of them at times have issues. Others play fine. I am trying to pinpoint exactly what files are having the issue.

No

DirectPlay

Nope, synced with Bazarr (work perfectly on actual jellyfin-media-player).

danielwikander commented 7 months ago

Do we have any idea what could be causing this, or where in the codebase one could start looking? This issue is bothering me so much I might try looking into it myself, but I'm new to the project so any guidance here would be appreciated.

Migs3 commented 4 months ago

Is this still being looked into? This is one of the most annoying things "wrong" with jellyfin. It doesn't seem like a difficult concept to simply use a single time value for displaying the video and subtitles. I don't know the underlying code or anything but it seems like the only way this could be a problem is if there are multiple time values that are needing to be kept in sync when there really should only be one time value when serving content.

Not only the above, but on a related note there should be a menu option in the gui/web to set a subtitle sync time value (+/-). A simple method to time shift the subtitles ahead or behind slightly. Not only would that help negate some of the frustration of this issue, but some subtitles are just slightly off when you have different media/subtitle releases that don't quite match up.

dmitrylyzo commented 4 months ago

Is this still being looked into? This is one of the most annoying things "wrong" with jellyfin. It doesn't seem like a difficult concept to simply use a single time value for displaying the video and subtitles. I don't know the underlying code or anything but it seems like the only way this could be a problem is if there are multiple time values that are needing to be kept in sync when there really should only be one time value when serving content.

The original issue has been fixed by https://github.com/jellyfin/jellyfin-web/pull/4385 If you have steps to reproduce your case, post them (what client, what subtitles type, etc.). Maybe even open a new issue (it may be different from the original one).

Not only the above, but on a related note there should be a menu option in the gui/web to set a subtitle sync time value (+/-). A simple method to time shift the subtitles ahead or behind slightly. Not only would that help negate some of the frustration of this issue, but some subtitles are just slightly off when you have different media/subtitle releases that don't quite match up.

It is in the Player Settings (gear button) / Subtitle Offset.

Ellypax commented 4 months ago

Hello, I'm experiencing ongoing issues with Jellyfin 10.8.13 running on Linux Docker x64. Specifically, I'm encountering varying subtitle offsets depending on the file. This problem persists across multiple files sourced from different origins. Each time, the subtitles fail to synchronize properly, and notably, the desynchronization tends to increase as the movie progresses.

This issue occurs specifically with integrated subtitles when accessed via Jellyfin web interface. I've been primarily using Safari on MacOS 14.4 and iOS 17.4.1 for playback.

Interestingly, when I utilize Jellyfin for Android TV with libVLC, there are no subtitle synchronization issues. Additionally, downloading the file and playing it on my computer also does not reproduce the problem.

Migs3 commented 3 months ago

The original issue has been fixed by #4385 If you have steps to reproduce your case, post them (what client, what subtitles type, etc.). Maybe even open a new issue (it may be different from the original one).

It is in the Player Settings (gear button) / Subtitle Offset.

Maybe this is specific to the Android TV client, It's mainly used on our Firestick at home and my GF insists on constantly bashing my "new media server" due to subtitles never being correct and that she always has to turn them off.

cthulhuz commented 3 months ago

Can confirm that this bug is present on Samsung TV (Tizen) running jellyfin-web. The subtitles work fine on other devices and players. Bug occurs when seeking in some files that are being direct played with SRT subtitles. Direct playing a file from the beginning works fine until you seek in it.

joost00719 commented 3 months ago

d players. Bug occurs when seeking in some files that are being direct played with SRT subtitles. Direct playing a file from the

Same here. VLC player also isn't a proper fix. It does fix the sync issues, but the content will be resized to fit the screen, and since almost no movie is 16:9, it will stretch almost all content. Also, it suffers from stutters and audio pops (chrome cast with android tv. version "jellfyin-androidtv 0.16.7 release").

thornbill commented 3 months ago

AndroidTV doesn't use web, so that's a different story.

As Dmitry already pointed out, if you are having issues with the Android TV app this is not the correct place.

random-human-being commented 2 months ago

I have also just experienced it in 10.9 with SRT subs, after skipping ahead. The video was being transcoded.

Switching to PGS subs forced them to be burned into the video, fixing the issue.

SpyrosRoum commented 1 month ago

I've started experiencing this on 10.9, while before (<10.9) it only happened on mobile clients, now it happens on the web too.
Is there any possible workaround or anything? It's extremely annoying, especially since the only "fix" I have found is to download the video locally

Kusefiru commented 1 month ago

I am noticing similar behavior on jellyfin-web 10.9.2 with Firefox 126.0. The subtitle synchronisation is correct at first, but get worst with time or when jumping in the file.

I have tested both embedded ASS subs and external SRT subs.

If I switch to Edge (125.0.2535.51), the subtitles are correctly synced. This is also the case when playing the video file directly on my computer, so I believe the subtitles in themselves are fine.

fungos commented 1 month ago

Same issue here with Roku client, LGTv client and FireTV client.

srt subtitles, some start correct and desync as it progress, most desync after a pause-quit-resume, some are just completely desync from start. It was ok with server 10.8.something, but 10.9 is becoming almost unusable unfortunately, forcing me back to Plex to be able to watch almost anything, which means the subtitles are perfect and the problem is with Jellyfin, not sure if entirely server side or both sides.

SpyrosRoum commented 1 month ago

Something I've noticed and helps a little is that using the https://github.com/jellyfin/jellyfin-vue web client seems to fix the issue for items that have de-synced on the main web client. Unfortunately this is not a complete solution since the vue client still misses features, but it saves me from having to download anything that I was watching and is broken...

deniax2 commented 1 month ago

Interestingly enough, for me using Swiftfin always has synced subtitles. Doesnt matter if i scrub throught the movie or change the language of the sub.

When I switch to my laptop with Chrome or Firefox, the subtitles are always desynced. I can sync them manually, but the moment I fast-forward or reverse, the subs are de-synced again.

So it really seems for me a client issue

viown commented 1 month ago

Interestingly enough, for me using Swiftfin always has synced subtitles. Doesnt matter if i scrub throught the movie or change the language of the sub.

When I switch to my laptop with Chrome or Firefox, the subtitles are always desynced. I can sync them manually, but the moment I fast-forward or reverse, the subs are de-synced again.

So it really seems for me a client issue

This is because Swiftfin direct-plays nearly all types of media without issues, while Chrome and Firefox have a more limited codec support which in your case is leading to direct-streaming or transcoding (which is what causes this issue).

This issue is almost purely server-side because the server cannot deliver the exact timestamp the client is asking for. Instead, the server will seek back to the earliest keyframe (which can be anywhere from 5-8 seconds wide) causing the de-sync.

Although I'm sure this could be fixed with some new API introductions where the server is able to tell the client the exact timestamp it's seeking to, but that requires explicit client support and there could be some other complications I'm missing. Maybe @gnattu could explain?

SpyrosRoum commented 1 month ago

@viown what you are saying is interesting but it raises the question, why did it become a bigger problem with v10.9? Something must have changed in the backend and perhaps it's possible to revert or fix that without breaking/client changes

gnattu commented 1 month ago

why did it become a bigger problem with v10.9

it is not. This issue was created for 10.8 isn't it?

SpyrosRoum commented 1 month ago

The issue was created for 10.8, but it didn't happen for me (on firefox) until 10.9, and based on the activity on the issue I believe other experienced the same.

To be clear I had experienced it before on the mobile (android, default player), but never on firefox.

creekdrops commented 3 weeks ago

Not sure if this issue is related, but I've noticed that when accessing subtitles over HLS using Apple's native player in Mac and iOS, the subtitles are extremely delayed by about 12 seconds.

finnurthorisson commented 1 week ago

Also happening for me with 10.9.6 and 10.9.7 in Firefox but did not happen in 10.8.10

Transcoded (HEVC->H264) in browser is desynced but in app (Direct playing) it works like normal. I've also had it occasionally replay a short chunk in browser which usually triggers it to desync. Desync persists when leaving and resuming