Is there any chance of activating caption without streaming video?

[zenodallavalle]

I'd like to get caption of my videos that I save locally, I don't use streaming functionality. BTW Thank you very much for developing this tool, it's amazing.

[raught]

It already sort of does, if you select Desktop Audio from the caption source but it works really bad for me. Misses entire sentences from the video, seems to just throw out a couple of words whenever it feels like. It's definitely made for streaming.

[ratwithacompiler]

@zenodallavalle Hi, sorry didnt see that earlier but if you set it to output to "Recordings" in settings it should caption OBS local recordings just like it would with streams.

[concretegargoyle]

I'm having the same issue. My local recording file doesn't have the captions. I tried mkv and mp4. Should the captions end up in the video or in a separate caption file? I feel like there is a step I'm missing.

[zenodallavalle]

@ratwithacompiler

@zenodallavalle Hi, sorry didnt see that earlier but if you set it to output to "Recordings" in settings it should caption OBS local recordings just like it would with streams.

Maybe I'm missing something but the only thing that I can get is a txt file with a timestamps and captions. I was hoping to get subtitles built in mkv or mp4 file or at least a .srt file.

[zenodallavalle]

It already sort of does, if you select Desktop Audio from the caption source but it works really bad for me. Misses entire sentences from the video, seems to just throw out a couple of words whenever it feels like. It's definitely made for streaming.

For me it works quite fine. I'm using it to get captions of some zoom meetings. The captions are quite ok except for specific terms. It very depends from relators' microphones' quality and from their voices.

[btbonval]

I'm also trying to do offline captions with recording rather than streaming, with no success. I'm running v0.14b on Windows with OBS version 25.0.8 all 64-bit.

Here are the steps I have taken:

• Settings > General
  • Caption Source: my audio source
  • Caption When: When Caption Source is streamed (no choice for when caption source is recorded)
  • Output To: I've tried literally every option here and it doesn't seem to make a difference.
• Settings > Transcripts
  • Save Transcripts checked.
  • Save For: Local Recordings Only (also tried Streams & Local Recordings)
  • Format: srt
• Captioning Enabled checked.

The preview shows text being transcribed as I run audio through the source, looks good. No srt file output.

I click "Start Recording". Preview continues showing transcription just fine. No srt file output. I click "Stop Recording". No srt file shows up.

I setup a custom stream to an rdp URI that goes nowhere. I click "Start Streaming". Preview continues showing transcription just fine. srt file shows up, but is empty. I'm getting random alerts from OBS that it lost the stream (because the rdp URI isn't a proper stream destination). I click "Stop Streaming". srt file gets content, but there are several skips of many seconds.

Hi, sorry didnt see that earlier but if you set it to output to "Recordings" in settings it should caption OBS local recordings just like it would with streams.

I'm trying to follow those instructions, but I'm clearly doing something wrong. What settings should I change so that this transcribes to an srt file when I click "Start Recording" ?

[ratwithacompiler]

I'm trying to follow those instructions, but I'm clearly doing something wrong. What settings should I change so that this transcribes to an srt file when I click "Start Recording" ?

That's weird, those transcript settings look perfectly fine and should save an srt file if you see captions in the preview or dock, no stream should be necessary.

Can you check your log file for errors OBS -> Help -> Logs -> View current logs or upload it?

[btbonval]

Thanks for the quick response.

I turned on the preview, began the audio, saw CC in the preview. Screenshot is captionpreview.PNG. No srt file output.

I then clicked start recording, saw CC in the preview. I also opened the settings (but didn't change them) so they were visible in captionpreview2.PNG. No srt file.

Attached both screenshots and the log.

2020-08-17 17-24-06.txt

[ratwithacompiler]

ohh weird 17:26:37.831: transcript_writer_loop find_transcript_filename error: couldn't get recording path

looks like it fails because it cant get the name of the current recording, I'll have to look into what's causing that but in the meantime you can set "Transcript filename" to the recording_datetime setting instead and it should hopefully work.

[btbonval]

I saw that in the log as well. Strangely, the file did get created properly when I did "Start Streaming" the one time, but maybe that is coincidence.

I will try that setting and get back with a positive/negative result. Have to take a break to work on other tasks for now though.

[ratwithacompiler]

Ohh nice I can reproduce that error now with recording set to Custom Output (FFmpeg) which I think is what you're using too based on the log. I didnt test with that, seems that works a bit differently, can fix that now, thanks!

[btbonval]

@ratwithacompiler Super! ~Glad I took the screenshots. Not sure how I missed that when detailing the options I chose.~ woops that wasn't in the screenshots

I wonder if others in this thread might have been getting hit or miss results based on settings like that.

I changed Recording Filename to recording_transcript_date.srt and it appears to be created when pressing "Start Recording". Looks like a solid workaround. Thanks!

[stewart1969]

I do not stream either, but want captions on my local recordings. Its all setup and working - the dock, the captioning, the saving of the .srt file. Yay! And when I replay my recorded video I simply switch the captions on and it shows the captions, so all fab! However, is it possible to put the captions straight onto the OBS so that the captions are actually 'burned' into the recording 'live' as I speak? I teach English and it would be good to have the captions on the actual video (rather than optionally selected by the viewer)

Currently, the captions only appear in the dock, not on my OBS Preview or OBS Program, so they don't record to the video because they are not on the screen. @ratwithacompiler Any ideas? Am i doing it wrong?

[ratwithacompiler]

@stewart1969 You can get burned in captions via text sources. Just add a text source to your scene(s) in OBS and then in the caption settings under Open Captions enable those and select your text source and it will write the captions into that text source so they will be directly in the video. That way you can also fully customize the positioning and look of them via the font/size/color options OBS has for text sources.

[Aigeec]

On a related note I'd like to do captioning using just the virtual camera? I'd also like to do translations. If you point me in the right direction would be happy to (try and) submit a PR.

[ratwithacompiler]

@Aigeec

Are you using text source open captions with the Virtual Camera output? The plugin generally only captions when OBS is streaming or recording or the preview window is open to not waste resources while people just have OBS open. Now with the new Virtual Camera output in OBS I should caption when that is running and Open Captions are enabled as well, I hadn't even thought about that yet, I can add that, thanks!

In the meantime to get captioning without streaming/recording you can just hit Show Preview and it will caption as long as e preview window is open somewhere.

Re translations I'm not really sure, I was looking into that a while ago and was hoping that https://www.ngi.eu/ngi-projects/bergamot/ would become a viable offline open source translation option eventually but it's not really there yet I believe. So for now the only real option I can see would be any of the online APIs of one of the usual cloud providers, but those are not cheap and would require everyone using it individually to sign up with said cloud provider and get their own API key (which can be surprisingly involved process for some of the providers too)

There's also the general problem of accuracy since Speech to Text by itself is far from perfect (and depending on the speaker can borderline terrible already), then adding machine translations on top of that and the amount of people for which that's actually practically works well enough to be useful for would probably not be that high I think.

So I'm not really sure if it makes sense to include that in the plugin in general. Most people use the plugin for Twitch with the native CEA 608 output in OBS that embeds it into the streams/recordings directly but CEA 608 is limited to 4 short lines of plain text without foreign char support so for that at best you could translate into a single western language replacing the original captions which limits the usefulness quite a bit for the embedded output.

For transcripts of streams/recordings it would probably also make more sense to just translate the transcript files by themselves afterwards (would also be a lot cheaper with the pricing model of the cloud translation APIs compare to constant calls with small chunks and lots of duplication needed for live translation).

For Open Captions burned into the video (or other possible ways of outputting captions) I don't think natively integrating that into a OBS plugin like this is the best option. The main advantage of a native plugin is the direct access to OBS internals and the native output but for just open captions you don't need any of that. Like for example there's websites like https://webcaptioner.com/ which lets you do open captions and just embed it as browser source in OBS and OBS also has plugin APIs for python and LUA that lets you do a lot.

So basically in short I think for getting captioning and translation to use for open captions burned into the video, transcripts, or any other possible high level outputs (like youtube live has an caption ingest API endpoint I think) you're probably much better off making something standalone either as website, OBS python plugin or maybe a standalone desktop tool in any higher level language than implementing it in C++ in a native plugin like this.

Making a basic website that uses the JS SpeechRecognition API and translating the output via any of the ready made js API libraries for the cloud providers or making a OBS python plugin that gets the mic audio and captions it and translates it using the google APIs python lib to write it to a OBS text source would be sooo much easier than doing all that via C++. Last time I tried setting up multiplatform automatic builds to use the Google GRPC speech recognition libs for the plugin took me like a week ...

I'm not great at keeping things short but basically I don't think doing translations within the plugin makes much sense currently.

[Aigeec]

@ratwithacompiler

Thanks so much for the time you have put into that answer.

The problem I am trying to solve is essentially real-time translating zoom meetings between office teams that are multi-lingual. Between accents and everything even those of us speaking English natively could sometime use subtitles.

I saw webcaptioner and it works we'll be doesn't translate. There is another site https://speechlogger.appspot.com/ that I think has probably implemented something along the lines of what you have suggested but is a paid service longer than 10 minutes it seems.

I will take a look at your suggested options.

Thanks again for your great response.

[kevinsullivan]

@stewart1969 You can get burned in captions via text sources. Just add a text source to your scene(s) in OBS and then in the caption settings under Open Captions enable those and select your text source and it will write the captions into that text source so they will be directly in the video. That way you can also fully customize the positioning and look of them via the font/size/color options OBS has for text sources.

Thank you Works. Should document, as this is a capability a lot of people will want to use.

[kevinsullivan]

@stewart1969 You can get burned in captions via text sources. Just add a text source to your scene(s) in OBS and then in the caption settings under Open Captions enable those and select your text source and it will write the captions into that text source so they will be directly in the video. That way you can also fully customize the positioning and look of them via the font/size/color options OBS has for text sources.

Thank you Works. Should document, as this is a capability a lot of people will want to use.

Strange, worked for me at first, but the second time I tried, text is reversed on my screen. Selecting reversed text widget, right clicking, selecting transform, and flip horizontal fixed it.

[nosys70]

hello I installed latest version plugin on unbuntu 20.04, OBS ver 27+ plugin seems to be there, i can configure it, but i got no caption in the windows and no SRT file. there are several source listed in the dropdown list for source. Mainly i see the input for video+audio (like presenter1) , but i also see the generic audio track (1 to 6). Is it ok to select the video+audio stream ? (i use RTMP source). i tried to select a dummy stream on my rtmp server and start streaming and recording (mkv) , but it does not help. (video file is here, but no srt files) I am not sure all ressource needed are installed, but it seems there is no error obviously visible. Any idea ? what to check ? Thanks for help