Subtitles out of sync in VOD streams with commercials.

[mediaminister]

VTM GO uses separate WEBVTT subtitle files along with the MPEG DASH streams, but subtitle timings in the WEBVTT files don't take into account the commercial breaks. Subtitles are displayed out of sync after a commercial has been shown.

For instance, these are the subtitles for 13 Geboden, episode 2 (Catalog - Vlaamse Series) https://dvt-subtitles.persgroep.be/13GEBODEN2_1F633728_272ef5ea-0eaa-4b0a-9a8f-b72ba8d7707e.vtt

VTM GO provides the exact durations of the commercials in a non-standard json file that also contains a master_m3u8 property containing a standard MPEG-DASH manifest url that is openend by InputStream Adaptive. The format of the a non-standard json file url is:

https://dcs-vod.apis.anvato.net/vod/p/manifest.mpd?<querystring>

An example (13 Geboden, episode 2) of the contents of this json file can be found here: manifest.txt

I conclude that the way VTM GO is handling subtitles is not part of an official streaming standard and therefore cannot be implemented in InputStream Adaptive.

This problem can be fixed by reading the commercial durations from the json and delay the subtitles accordingly. We should investigate how Kodi supports manipulating subtitle timings. Worst case scenario we have to edit all timings of the WEBVTT file, save this edited file and load this into the Kodi player.

[michaelarnauts]

I also remember seeing subtitles in the dash manifest itself. I don't have an example however. Maybe VTM GO uses two ways to present subtitles, trough the manifest, or externally?

[peak3d]

`

https://dpp-anvato-live.akamaized.net/eu/vod/19/09/04/10046235/B52E172C2A14EB35690C331AE3C65644F8737BF5927BE83.mp4?hdntl=exp=1568068972~acl=/*~hmac=518b817a4c6031b5322805d3d10e413002cf5585c8707edaa85e91d764aa22ee ` This is from first VOD in letter A (A-Z section), should be supported already from inputstream.adaptiive

[mediaminister]

I forgot about the subtitle stream in the MPEG-DASH Manifest. I don't know if this stream contains the right subtitles. I didn't investigate this yet.

[mediaminister]

Maybe VTM GO uses two ways to present subtitles, trough the manifest, or externally?

That's right, there is an external WEBVTT file and there are internal segmented WEBVTT subtitles. I can't extract these internal subtitles with ffmpeg but I can watch them with https://developer.jwplayer.com/tools/stream-tester/ JW Player displays the subtitles during the first part of the program but stops displaying them after the commercial break. But I'm convinced there is nothing wrong with the internal subtitles, this is most certainly just a bug in JW Player.

The internal segmented WEBVTT subtitles can be handled by InputStream Adaptive, but currently the internal subtitles are not displayed right, so there is a bug or implementation is incomplete.

Kodi itself has a good WEBVTT implementation for external subtitle files but we need to delay the subtitles if a commercial is played.

So, there are two possibilities to fix the subtitle problem.

[mediaminister]

I made an experimental fix, editing the timestamps from the external WEBVTT file was the easiest solution to implement.

During testing I experienced that the Periods implementation in InputStream Adaptive is not ready for a release, there is a problem with the timings when seeking forward, switching between periods goes not smoothly, audio or video freezes for a short time when the next period is started,...

[peak3d]

@mediaminister what should I do with your last post? Any action required?

[mediaminister]

@peak3d I'll explain the problems I experienced in more detail, I didn't have the time yet to investigate and document this more thoroughly:

1. Consecutive periods problem: when a next period starts, audio and video are not in sync, audio or video freezes but after a couple of seconds this normalizes.
2. Seeking forward problem: when seeking forward, the Kodi subtitles get out of sync. Seeking backwards fixes this problem.
3. Past period plays again: after an advertisement break, the previous Period plays again, seeking forward or backwards fixes this (but problem 2 persists)

I will open an issue with more information of these three problems in the InputStream Adaptive repo next weekend when I find the time. But if you can take a look at this before, feel free to do so.

[michaelarnauts]

@peak3d I've also noticed this. After a period, the audio goes out of sync and the video stalls until it "catches up". This could take up to 10 seconds.

[peak3d]

@mediaminister could it be, that the external subtitle stream leads to your issues? Player tries to sync all streams, if one is odd, it causes problems. I will not know how you try to add external subtitles without knowing how inputstream.adaptive works regarding PTS b.t.w. IMO external subs are currently impossible.

[peak3d]

For the future: pls. provide a debug log with most probably only the issue played. Pls. don't provide logs with several issues. Without debug log / detailed information I'll not start any action. Pls. kep in mind that I spend many more hours for your feature request, so it should be possible that you spend some minutes to produce a clean, unique debug log. Thx.

[mediaminister]

The external subtitles are handled in the VTM GO add-on and added directly to Kodi Player with listitem.setSubtitles InputStream Adaptive is not involved in this functionality. It's possible there is a problem with the external subtitles, but I experienced also a clear difference between seeking forward and seeking backwards. Problem 2 only occurs when seeking forward.

As I said, I should investigate these problems more thoroughly and I will create issues when this is done. I don't expect you to take action with this short and unclear explanation.

You can be sure I appreciate all your efforts. Thank you!

[peak3d]

@mediaminister stream sync is handled by PTS, currrently PTS in inputstream.adaptive is starting at 0 for every period. Because of this there is IMO no real chance for you to provide external subs. PTS = PresentationTimeStamp

[dagwieers]

could it be, that the external subtitle stream leads to your issues? Player tries to sync all streams, if one is odd, it causes problems.

@michaelarnauts Do we want to provide an option related to subtitles, that influences the use of subtitles for the VTM GO add-on specifically? Could also be useful for debugging. If so, I will create an issue/feature request.

[michaelarnauts]

Hmm, I don't think so. If Kodi can find out the right language of the subtitle, it can automatically enable or disable the subtitles depending on your Kodi settings.

The modifying of the subtitles should just work in the background. No reason to turn it of in the settings.

[peak3d]

@dagwieers I can reproduce the issue here, interestingly video only / audio only seems to work without issues (you can select audio / video only in inputstream.adaptive settings) Beside this the PTS of A and V seems to match, so its either a problem due to subs, or something inside kodi

[peak3d]

@michaelarnauts I don't trust that Kodi does not use the subs internally if you provide them. To be sure that subs are not the issue, it would be helpful to have a version where no external subs are passed. Maybe you can simply tell me the line where this is done, so I can try by myself.

[dagwieers]

@michaelarnauts We do this in VRT NU settings: Show subtitles when available I think it would be useful to have this switch as well in VTM GO.

[peak3d]

I pushed a commit to the IChapter kodi PR. Pls. test if this solves your issues when changing chapters.

[michaelarnauts]

Hmm, but this is a setting in Kodi under Player > Language. You can set subtitles to None there to disable them, or set it to Dutch if you like them. I find the override in VRT NU confusing, and I don't really get the point. Enlighten me :)

[michaelarnauts]

@peak3d for debugging, sure. The line is here: https://github.com/michaelarnauts/plugin.video.vtm.go/blob/13ee189917d0731f6ad715040bc2abf46a96f190/resources/lib/plugin.py#L527

I guess you also want to comment the next line.

[mediaminister]

I find the override in VRT NU confusing, and I don't really get the point. Enlighten me :)

This is a setting especially for hearing impaired people and non-native speakers who always need subtitles to understand Dutch spoken programs. This ensures that these people only have to switch on subtitles once.

Proposals to improve this setting are welcome at https://github.com/pietje666/plugin.video.vrt.nu/issues

[michaelarnauts]

@mediaminister sure, but isn't this exactly what that Kodi setting does? Again, I might be wrong here. I'm just trying to figure out the use case.

[mediaminister]

I pushed a commit to the IChapter kodi PR. Pls. test if this solves your issues when changing chapters.

@peak3d I tested your latest changes ( InputStream Adaptive master branch and Kodi master branch with PR-16581) and this greatly improves Multiple Periods support. It fixes 2 of the 3 problems I mentioned earlier in this thread. Many thanks for fixing these problems!

Unfortunately, there is still one problem with the synchronization of subtitles when you seek forward with the mouse pointer on the Kodi progress bar. I documented this problem in the InputStream Adaptive repo like you suggested:

For the future: pls. provide a debug log with most probably only the issue played.

@peak3d I created an issue for the "seeking forward problem" that is still there with PR-16581: https://github.com/peak3d/inputstream.adaptive/issues/324

[peak3d]

@mediaminister I pushed changes for subs in mp4 container yesterday (or today, I forgot) For me the vtm-go internal subs are working now

[mediaminister]

@peak3d Many thanks for your efforts, I can confirm the internal subs are working, but I suspect VTM GO doesn't provide the internal subtitles correctly. I only get subtitles in the first part of the program. (13 Geboden - Aflevering 2)

So, we still need the external subtitles to get subtitling for the full program.

[peak3d]

Yes, seems that they have the complete subtitle file in part 1, I don't see any nice solution how to solve it

[michaelarnauts]

The external subtitles seem to be fine, the internal don't seem to be used. Thanks for the fix @mediaminister!