Add subtitle support?

[trcrsired]

I want to see something like song lyrics to play when the voice is played.

[mrthinger]

Could you better describe what you're imagining? Potentially mock up a visual? The read text is actually already held in the soundData table, so its just a matter of displaying it.

[Astraorion]

Absolutely great suggestion! I would also love to have subtitles

[Astraorion]

Could you better describe what you're imagining? Potentially mock up a visual? The read text is actually already held in the soundData table, so its just a matter of displaying it.

I would suggest displaying it like regular subtitles in a movie. 1 or maximum 2 sentences at once. With option to activate or deactivate them.

[JohnEdwa]

Also the current playback box is already set up to be very similar how the addon Immersion does its text, that would be my first choice on how to display them. It might even come as a side product of implementing #37 with all the text splitting and syncing, just requires adding the ability to have VoiceOver do it for those of us who can't (1.12.1) or don't want to use Immersion.

[xIGBClutchIx]

Yeah, it would be like the talking head frame like Retail.

[mrthinger]

This is a huge thing to take on and do correctly. It would add value, but I dont have the bandwidth to take it on anytime soon.

There was some talk a while back in the discord about generating offsets of the spoken text from the audio files using that to sync to the shown text. The small addon window that shows can't fit the entire quest into it at the same time. If you're looking to see the static text in its entirety, you could open the quest log and follow it yourself.

Maybe someone could make a proof of concept where the text isnt synced to the audio, but is based on text length.

[mrthinger]

I'm moving over @MattS8 's useful thoughts on how we could approach this problem from #37, which discussed adding subtitle support via integration with the Immersion addon.

Timestamping transcripts of the audio files definitely seems doable. There are a couple of things that I would like to look into more (or really anyone can) first. Ideally, the following three things are needed for complete harmony between the two addons:

1. Voiceover should understand how Immersion breaks text down into chunks
2. Voiceover should know when Immersion moves between text chunks
3. Voiceover should be able to control Immersion paging through dialog in time with the audio

1. Understand how Immersion breaks text down into chunks:

I've already started looking through Immersion's code and created a super rough makeshift set of functions that emulates how Immersion decides to break up text. Using this, we should be able to simply run the transcripts through and save the generated breakpoints (or, simply have it run at runtime).

2. Know when Immersion moves between text chunks

We need a way to hook into Immersion functions that cycle through text chunks. If, for example, a player manually skips to the next chunk of text, the audio should skip to the first word associated with the next chunk. I admit I know very little of WoW Addon development, however I don't think this is possible unless the Immersion addon registers a custom message (i.e. through AceEvents). Therefore, we would most likely have to request the addition of custom messages to Immersion's paging functions.

3. Control moving through dialog

Immersion controls text paging via a customizable time-based setting. It's static and it can even be turned off completely. To get Immersion and VoiceOver to be completely synced, we would have to send event messages at the breakpoint timestamps, and Immersion would then need to act on them by paging the text. This would most likely mean a setting within Immersion that would enable this new feature.

If I'm way overengineering this, lemme know.

Originally posted by @MattS8 in https://github.com/mrthinger/wow-voiceover/issues/37#issuecomment-1501507193