search

ThioJoe / Auto-Synced-Translated-Dubs

Automatically translates the text of a video based on a subtitle file, and then uses AI voice services to create a new dubbed & translated audio track where the speech is synced using the subtitle's timings.
GNU General Public License v3.0
1.6k stars 158 forks source link

Problem using the Batch processing (Azure) #72

Closed AdnielSsj closed 11 months ago

AdnielSsj commented 1 year ago

When I try to Batch Process with Azure, it gives me this error.

Translating text using Google...

Waiting for Azure batch synthesis job to finish. Status: [NotStarted]
ERROR: Batch synthesis job failed!
Reason:OK
Traceback (most recent call last):
  File "F:\Auto-Synced-Translated-Dubs-0.14.1\main.py", line 281, in <module>
    process_language(langData, processedCount, totalLanguages)
  File "F:\Auto-Synced-Translated-Dubs-0.14.1\main.py", line 267, in process_language
    individualLanguageSubsDict = audio_builder.build_audio(individualLanguageSubsDict, langDict, totalAudioLength, config['two_pass_voice_synth'])
  File "F:\Auto-Synced-Translated-Dubs-0.14.1\Scripts\audio_builder.py", line 76, in build_audio
    rawClip = AudioSegment.from_file(value['TTS_FilePath'], format="mp3", frame_rate=int(config['synth_sample_rate']))
KeyError: 'TTS_FilePath'

F:\Auto-Synced-Translated-Dubs-0.14.1>

Just for information, I am on the standard subscription, not the free one. So, I tried to deactivate the Azure Batch process, and it shows me the error:

Traceback (most recent call last):
  File "F:\Auto-Synced-Translated-Dubs-0.14.1\main.py", line 281, in <module>
    process_language(langData, processedCount, totalLanguages)
  File "F:\Auto-Synced-Translated-Dubs-0.14.1\main.py", line 267, in process_language
    individualLanguageSubsDict = audio_builder.build_audio(individualLanguageSubsDict, langDict, totalAudioLength, config['two_pass_voice_synth'])
  File "F:\Auto-Synced-Translated-Dubs-0.14.1\Scripts\audio_builder.py", line 76, in build_audio
    rawClip = AudioSegment.from_file(value['TTS_FilePath'], format="mp3", frame_rate=int(config['synth_sample_rate']))
  File "C:\Users\Brown\AppData\Local\Programs\Python\Python310\lib\site-packages\pydub\audio_segment.py", line 773, in from_file
    raise CouldntDecodeError(
pydub.exceptions.CouldntDecodeError: Decoding failed. ffmpeg returned error code: 1

Output from ffmpeg/avlib:

ffmpeg version 6.0-full_build-www.gyan.dev Copyright (c) 2000-2023 the FFmpeg developers
  built with gcc 12.2.0 (Rev10, Built by MSYS2 project)
  configuration: --enable-gpl --enable-version3 --enable-static --disable-w32threads --disable-autodetect --enable-fontconfig --enable-iconv --enable-gnutls --enable-libxml2 --enable-gmp --enable-bzlib --enable-lzma --enable-libsnappy --enable-zlib --enable-librist --enable-libsrt --enable-libssh --enable-libzmq --enable-avisynth --enable-libbluray --enable-libcaca --enable-sdl2 --enable-libaribb24 --enable-libdav1d --enable-libdavs2 --enable-libuavs3d --enable-libzvbi --enable-librav1e --enable-libsvtav1 --enable-libwebp --enable-libx264 --enable-libx265 --enable-libxavs2 --enable-libxvid --enable-libaom --enable-libjxl --enable-libopenjpeg --enable-libvpx --enable-mediafoundation --enable-libass --enable-frei0r --enable-libfreetype --enable-libfribidi --enable-liblensfun --enable-libvidstab --enable-libvmaf --enable-libzimg --enable-amf --enable-cuda-llvm --enable-cuvid --enable-ffnvcodec --enable-nvdec --enable-nvenc --enable-d3d11va --enable-dxva2 --enable-libvpl --enable-libshaderc --enable-vulkan --enable-libplacebo --enable-opencl --enable-libcdio --enable-libgme --enable-libmodplug --enable-libopenmpt --enable-libopencore-amrwb --enable-libmp3lame --enable-libshine --enable-libtheora --enable-libtwolame --enable-libvo-amrwbenc --enable-libilbc --enable-libgsm --enable-libopencore-amrnb --enable-libopus --enable-libspeex --enable-libvorbis --enable-ladspa --enable-libbs2b --enable-libflite --enable-libmysofa --enable-librubberband --enable-libsoxr --enable-chromaprint
  libavutil      58.  2.100 / 58.  2.100
  libavcodec     60.  3.100 / 60.  3.100
  libavformat    60.  3.100 / 60.  3.100
  libavdevice    60.  1.100 / 60.  1.100
  libavfilter     9.  3.100 /  9.  3.100
  libswscale      7.  1.100 /  7.  1.100
  libswresample   4. 10.100 /  4. 10.100
  libpostproc    57.  1.100 / 57.  1.100
[mp3 @ 000001d67b755900] Failed to read frame size: Could not seek to 1026.
workingFolder\1.mp3: Invalid argument

F:\Auto-Synced-Translated-Dubs-0.14.1>python main.py

And it ends up happening that Azure creates 0kb .mp3 audio files, and the error that they had fixed in fix #17 reappears. For information, I am using the latest version of ASTD 0.14.1.

image

I tried changing add_line_buffer_milliseconds = 0 to add_line_buffer_milliseconds = 1. Because I saw it was a possible error cause, but it still didn't work, and the issue with empty audio files persists.

ThioJoe commented 11 months ago

I believe this happens when skip_synthesize is enabled in some circumstances. I'm able to reproduce in that situation. I'll have to investigate.

I figured out the issue. This happens when both skip_synthesize is True and batch_tts_synthesize is True.

The reason is that when skipping voice synthesis, it uses the files inside workingFolder, so it can only do this for a single language, not a batch of them.

I'll have to add a warning for this case.

ThioJoe commented 11 months ago

Should be fixed as of commit 411b5148632cd21c05acf171e5abc260b4607598 So it will be in the next release