Import Transcripts (and/or subtitles) into autoEdit3

[billdre]

Hi Pietro,

Thank you so much for your work on this project.

I have one quick question: Is there a way to import subtitles files/pre-made transcripts into AutoEdit3?

Regards, Bill

[pietrop]

Hi @billdre I used to have the option to import subtitles in autoEdit2 so theoretically it could be possible to add it back.

What use case do you have for which you need to import subtitles?

[billdre]

I'm editing videos in Thai language and current STT engines in AutoEdit3 do not support it. I'm thinking about training a Deepspeech model for Thai but it's almost impossible to my knowledge. Importing .srt seems to be the quickest way if I want to work with a foreign language in AutoEdit3. (I mainly use Paper Edits + Premiere Pro)

I'm looking at autoEdit2 code to find a way to move the feature to AutoEdit3. Is it easy or is it gonna be hard? (I'm a video editor with some very basic coding knowledge)

Thanks Pietro!

[pietrop]

Hi @billdre Yeah, I probably wouldn't train the deepspeech model, I honestly don't understand that myself + Mozilla announced they are not going to be support deepspeech any longer.

Re adding feature from autoEdit 2.

This is the main module that does the srt caption conversion OpenNewsLabs/autoEdit_2/lib/interactive_transcription_generator/transcriber/captions altho autoEdit3 uses this new json format, refrred to as DPE json (Digital Paper Edit) so the converted would need to be adapted.

DPE json is like this

{
  "words": [
    {
      "end": 1.63,
      "start": 1.41,
      "text": "So"
    },
    {
      "end": 2.175,
      "start": 1.63,
      "text": "tell"
    },
    ...
   ]
  "paragraphs": [
    {
      "speaker": "James Jacoby",
      "start": 1.41,
      "end": 4.88,
      "id": "0"
    },
    {
      "speaker": "James Jacoby",
      "start": 4.88,
      "end": 7.21,
      "id": "1"
    },
   ....
  ]
}

There's an earlier one I made a while back, that might work for this use case

pietrop/srtParserComposer - srt to word accurate time

This only gives the words. The "paragraphs" would still need to be interpolated, based on punctuation. eg every time there's a . in a word, that could mark start and end of paragraph time range, if that makes sense?

Adapting that module, for autoEdit3/DPE could be a good start.

Then to add the module to autoEdit3 it would be a matter of modifying the react client pietrop/digital-paper-edit-client) for adding support for an optional srt file in the React new transcript form pietrop/digital-paper-edit-client/src/Components/Transcripts/TranscriptForm.js and then in the ElectronAPIWrapper in the electron repository pietrop/digital-paper-edit-electron/src/ElectronWrapper/index.js#L115 support for parsing the srt, and saving to db.

I can provide more details on this last part to unpack it in more details if needed at a later stage.