Closed JaMeZ-B closed 1 year ago
I managed to get working word boundaries by requesting a style (Conrad only knows how to be cheerful…):
self.set_speech_service(
AzureService(
voice="de-DE-ConradNeural",
style="cheerful",
transcription_model='base' # this is not needed
)
)
Is this a general thing, i.e. do you always have to set a style
? – this seems at least counterintuitive to me.
Could it be working now because of transcription_model='base'
instead? AFAIR it should override the word boundaries returned by Azure.
Ideally, none of this should be necessary, unless there is something wrong with the output from Azure. The issue is most likely in TimeInterpolator
. I will look into this soon.
In the meanwhile, you could take look at media/voiceover/cache.json
to see if there is sth wrong with the timestamps.
Setting transcription_model='base'
does not have an effect on the Azure speech service afaik, due to https://github.com/ManimCommunity/manim-voiceover/blob/dc7050d988d71e3f33631988ca07dbc790a5b817/manim_voiceover/services/base.py#L81-L90C13
I just found out the what issue is, it was trivial. Releasing a new version with the fix now
v0.3.3 should fix this issue altogether, let me know if you run into any further issues.
Description of bug / unexpected behavior
The Azure voice
de-DE-ConradNeural
leads to warningsduring compilation. Timing using bookmarks is off in the resulting video.
Expected behavior
Correct timing via bookmarks as with the voice
en-US-AriaNeural
How to reproduce the issue
Use the voice
de-DE-ConradNeural
instead ofen-US-AriaNeural
inbookmark-example.py
Code for reproducing the problem
```py from manim import * from manim_voiceover import VoiceoverScene # from manim_voiceover.services.coqui import CoquiService from manim_voiceover.services.azure import AzureService class BookmarkExample(VoiceoverScene): def construct(self): # self.set_speech_service(CoquiService()) self.set_speech_service( AzureService( voice="de-DE-ConradNeural" # style="newscast-casual", ) ) blist = BulletedList( "Trigger animations", "At any word", "Bookmarks", font_size=64 ) with self.voiceover( text="""Manim-Voiceover allows you toAdditional media files
Images/GIFs
https://github.com/ManimCommunity/manim-voiceover/assets/7613071/21a761b9-9390-4be5-807b-df25fc817e23Logs
Terminal output
``` Manim Community v0.17.3 [05/12/23 12:21:32] WARNING TimeInterpolator received weird input, tracker.py:30 there may be something wrong with the word boundaries. DEBUG Animation with empty mobject animation.py:174 INFO Caching disabled. cairo_renderer.py:68 DEBUG List of the first few animation cairo_renderer.py:87 hashes of the scene: ['uncached_00000'] INFO Animation 0 : Partial movie scene_file_writer.py:527 file written in '/Users/bantje/manim-test/medi a/videos/480p15/partial_movie_ files/BookmarkExample/uncached _00000.mp4' INFO Caching disabled. cairo_renderer.py:68 DEBUG List of the first few animation cairo_renderer.py:87 hashes of the scene: ['uncached_00000', 'uncached_00001'] [05/12/23 12:21:33] INFO Animation 1 : Partial movie scene_file_writer.py:527 file written in '/Users/bantje/manim-test/medi a/videos/480p15/partial_movie_ files/BookmarkExample/uncached _00001.mp4' DEBUG Animation with empty mobject animation.py:174 INFO Caching disabled. cairo_renderer.py:68 DEBUG List of the first few animation cairo_renderer.py:87 hashes of the scene: ['uncached_00000', 'uncached_00001', 'uncached_00002'] INFO Animation 2 : Partial movie scene_file_writer.py:527 file written in '/Users/bantje/manim-test/medi a/videos/480p15/partial_movie_ files/BookmarkExample/uncached _00002.mp4' INFO Caching disabled. cairo_renderer.py:68 DEBUG List of the first few animation cairo_renderer.py:87 hashes of the scene: ['uncached_00000', 'uncached_00001', 'uncached_00002', 'uncached_00003'] INFO Animation 3 : Partial movie scene_file_writer.py:527 file written in '/Users/bantje/manim-test/medi a/videos/480p15/partial_movie_ files/BookmarkExample/uncached _00003.mp4' DEBUG Animation with empty mobject animation.py:174 INFO Caching disabled. cairo_renderer.py:68 DEBUG List of the first few animation cairo_renderer.py:87 hashes of the scene: ['uncached_00000', 'uncached_00001', 'uncached_00002', 'uncached_00003', 'uncached_00004'] INFO Animation 4 : Partial movie scene_file_writer.py:527 file written in '/Users/bantje/manim-test/medi a/videos/480p15/partial_movie_ files/BookmarkExample/uncached _00004.mp4' INFO Caching disabled. cairo_renderer.py:68 DEBUG List of the first few animation cairo_renderer.py:87 hashes of the scene: ['uncached_00000', 'uncached_00001', 'uncached_00002', 'uncached_00003', 'uncached_00004'] INFO Animation 5 : Partial movie scene_file_writer.py:527 file written in '/Users/bantje/manim-test/medi a/videos/480p15/partial_movie_ files/BookmarkExample/uncached _00005.mp4' INFO Caching disabled. cairo_renderer.py:68 DEBUG List of the first few animation cairo_renderer.py:87 hashes of the scene: ['uncached_00000', 'uncached_00001', 'uncached_00002', 'uncached_00003', 'uncached_00004'] [05/12/23 12:21:34] INFO Animation 6 : Partial movie scene_file_writer.py:527 file written in '/Users/bantje/manim-test/medi a/videos/480p15/partial_movie_ files/BookmarkExample/uncached _00006.mp4' INFO Caching disabled. cairo_renderer.py:68 DEBUG List of the first few animation cairo_renderer.py:87 hashes of the scene: ['uncached_00000', 'uncached_00001', 'uncached_00002', 'uncached_00003', 'uncached_00004'] [05/12/23 12:21:35] INFO Animation 7 : Partial movie scene_file_writer.py:527 file written in '/Users/bantje/manim-test/medi a/videos/480p15/partial_movie_ files/BookmarkExample/uncached _00007.mp4' INFO Caching disabled. cairo_renderer.py:68 DEBUG List of the first few animation cairo_renderer.py:87 hashes of the scene: ['uncached_00000', 'uncached_00001', 'uncached_00002', 'uncached_00003', 'uncached_00004'] [05/12/23 12:21:36] INFO Animation 8 : Partial movie scene_file_writer.py:527 file written in '/Users/bantje/manim-test/medi a/videos/480p15/partial_movie_ files/BookmarkExample/uncached _00008.mp4' DEBUG Animation with empty mobject animation.py:174 INFO Caching disabled. cairo_renderer.py:68 DEBUG List of the first few animation cairo_renderer.py:87 hashes of the scene: ['uncached_00000', 'uncached_00001', 'uncached_00002', 'uncached_00003', 'uncached_00004'] INFO Animation 9 : Partial movie scene_file_writer.py:527 file written in '/Users/bantje/manim-test/medi a/videos/480p15/partial_movie_ files/BookmarkExample/uncached _00009.mp4' WARNING TimeInterpolator received weird input, tracker.py:30 there may be something wrong with the word boundaries. WARNING TimeInterpolator received weird input, tracker.py:30 there may be something wrong with the word boundaries. INFO Caching disabled. cairo_renderer.py:68 DEBUG List of the first few animation cairo_renderer.py:87 hashes of the scene: ['uncached_00000', 'uncached_00001', 'uncached_00002', 'uncached_00003', 'uncached_00004'] INFO Animation 10 : Partial movie scene_file_writer.py:527 file written in '/Users/bantje/manim-test/medi a/videos/480p15/partial_movie_ files/BookmarkExample/uncached _00010.mp4' DEBUG Animation with empty mobject animation.py:174 INFO Caching disabled. cairo_renderer.py:68 DEBUG List of the first few animation cairo_renderer.py:87 hashes of the scene: ['uncached_00000', 'uncached_00001', 'uncached_00002', 'uncached_00003', 'uncached_00004'] INFO Animation 11 : Partial movie scene_file_writer.py:527 file written in '/Users/bantje/manim-test/medi a/videos/480p15/partial_movie_ files/BookmarkExample/uncached _00011.mp4' INFO Caching disabled. cairo_renderer.py:68 DEBUG List of the first few animation cairo_renderer.py:87 hashes of the scene: ['uncached_00000', 'uncached_00001', 'uncached_00002', 'uncached_00003', 'uncached_00004'] INFO Animation 12 : Partial movie scene_file_writer.py:527 file written in '/Users/bantje/manim-test/medi a/videos/480p15/partial_movie_ files/BookmarkExample/uncached _00012.mp4' DEBUG Animation with empty mobject animation.py:174 INFO Caching disabled. cairo_renderer.py:68 DEBUG List of the first few animation cairo_renderer.py:87 hashes of the scene: ['uncached_00000', 'uncached_00001', 'uncached_00002', 'uncached_00003', 'uncached_00004'] [05/12/23 12:21:37] INFO Animation 13 : Partial movie scene_file_writer.py:527 file written in '/Users/bantje/manim-test/medi a/videos/480p15/partial_movie_ files/BookmarkExample/uncached _00013.mp4' DEBUG Animation with empty mobject animation.py:174 INFO Caching disabled. cairo_renderer.py:68 DEBUG List of the first few animation cairo_renderer.py:87 hashes of the scene: ['uncached_00000', 'uncached_00001', 'uncached_00002', 'uncached_00003', 'uncached_00004'] INFO Animation 14 : Partial movie scene_file_writer.py:527 file written in '/Users/bantje/manim-test/medi a/videos/480p15/partial_movie_ files/BookmarkExample/uncached _00014.mp4' INFO Caching disabled. cairo_renderer.py:68 DEBUG List of the first few animation cairo_renderer.py:87 hashes of the scene: ['uncached_00000', 'uncached_00001', 'uncached_00002', 'uncached_00003', 'uncached_00004'] [05/12/23 12:21:38] INFO Animation 15 : Partial movie scene_file_writer.py:527 file written in '/Users/bantje/manim-test/medi a/videos/480p15/partial_movie_ files/BookmarkExample/uncached _00015.mp4' DEBUG Animation with empty mobject animation.py:174 INFO Caching disabled. cairo_renderer.py:68 DEBUG List of the first few animation cairo_renderer.py:87 hashes of the scene: ['uncached_00000', 'uncached_00001', 'uncached_00002', 'uncached_00003', 'uncached_00004'] INFO Animation 16 : Partial movie scene_file_writer.py:527 file written in '/Users/bantje/manim-test/medi a/videos/480p15/partial_movie_ files/BookmarkExample/uncached _00016.mp4' INFO Caching disabled. cairo_renderer.py:68 DEBUG List of the first few animation cairo_renderer.py:87 hashes of the scene: ['uncached_00000', 'uncached_00001', 'uncached_00002', 'uncached_00003', 'uncached_00004'] [05/12/23 12:21:39] INFO Animation 17 : Partial movie scene_file_writer.py:527 file written in '/Users/bantje/manim-test/medi a/videos/480p15/partial_movie_ files/BookmarkExample/uncached _00017.mp4' WARNING TimeInterpolator received weird input, tracker.py:30 there may be something wrong with the word boundaries. WARNING TimeInterpolator received weird input, tracker.py:30 there may be something wrong with the word boundaries. WARNING TimeInterpolator received weird input, tracker.py:30 there may be something wrong with the word boundaries. WARNING TimeInterpolator received weird input, tracker.py:30 there may be something wrong with the word boundaries. INFO Caching disabled. cairo_renderer.py:68 DEBUG List of the first few animation cairo_renderer.py:87 hashes of the scene: ['uncached_00000', 'uncached_00001', 'uncached_00002', 'uncached_00003', 'uncached_00004'] [05/12/23 12:21:40] INFO Animation 18 : Partial movie scene_file_writer.py:527 file written in '/Users/bantje/manim-test/medi a/videos/480p15/partial_movie_ files/BookmarkExample/uncached _00018.mp4' INFO Caching disabled. cairo_renderer.py:68 DEBUG List of the first few animation cairo_renderer.py:87 hashes of the scene: ['uncached_00000', 'uncached_00001', 'uncached_00002', 'uncached_00003', 'uncached_00004'] INFO Animation 19 : Partial movie scene_file_writer.py:527 file written in '/Users/bantje/manim-test/medi a/videos/480p15/partial_movie_ files/BookmarkExample/uncached _00019.mp4' INFO Caching disabled. cairo_renderer.py:68 DEBUG List of the first few animation cairo_renderer.py:87 hashes of the scene: ['uncached_00000', 'uncached_00001', 'uncached_00002', 'uncached_00003', 'uncached_00004'] INFO Animation 20 : Partial movie scene_file_writer.py:527 file written in '/Users/bantje/manim-test/medi a/videos/480p15/partial_movie_ files/BookmarkExample/uncached _00020.mp4' INFO Caching disabled. cairo_renderer.py:68 DEBUG List of the first few animation cairo_renderer.py:87 hashes of the scene: ['uncached_00000', 'uncached_00001', 'uncached_00002', 'uncached_00003', 'uncached_00004'] INFO Animation 21 : Partial movie scene_file_writer.py:527 file written in '/Users/bantje/manim-test/medi a/videos/480p15/partial_movie_ files/BookmarkExample/uncached _00021.mp4' INFO Caching disabled. cairo_renderer.py:68 DEBUG List of the first few animation cairo_renderer.py:87 hashes of the scene: ['uncached_00000', 'uncached_00001', 'uncached_00002', 'uncached_00003', 'uncached_00004'] INFO Animation 22 : Partial movie scene_file_writer.py:527 file written in '/Users/bantje/manim-test/medi a/videos/480p15/partial_movie_ files/BookmarkExample/uncached _00022.mp4' [05/12/23 12:21:41] DEBUG Animation with empty mobject animation.py:174 INFO Caching disabled. cairo_renderer.py:68 DEBUG List of the first few animation cairo_renderer.py:87 hashes of the scene: ['uncached_00000', 'uncached_00001', 'uncached_00002', 'uncached_00003', 'uncached_00004'] INFO Animation 23 : Partial movie scene_file_writer.py:527 file written in '/Users/bantje/manim-test/medi a/videos/480p15/partial_movie_ files/BookmarkExample/uncached _00023.mp4' INFO Caching disabled. cairo_renderer.py:68 DEBUG List of the first few animation cairo_renderer.py:87 hashes of the scene: ['uncached_00000', 'uncached_00001', 'uncached_00002', 'uncached_00003', 'uncached_00004'] INFO Animation 24 : Partial movie scene_file_writer.py:527 file written in '/Users/bantje/manim-test/medi a/videos/480p15/partial_movie_ files/BookmarkExample/uncached _00024.mp4' DEBUG Animation with empty mobject animation.py:174 INFO Caching disabled. cairo_renderer.py:68 DEBUG List of the first few animation cairo_renderer.py:87 hashes of the scene: ['uncached_00000', 'uncached_00001', 'uncached_00002', 'uncached_00003', 'uncached_00004'] INFO Animation 25 : Partial movie scene_file_writer.py:527 file written in '/Users/bantje/manim-test/medi a/videos/480p15/partial_movie_ files/BookmarkExample/uncached _00025.mp4' INFO Combining to Movie file. scene_file_writer.py:617 DEBUG Partial movie files to combine scene_file_writer.py:561 (26 files): ['/Users/bantje/manim-test/med ia/videos/480p15/partial_movie _files/BookmarkExample/uncache d_00000.mp4', '/Users/bantje/manim-test/medi a/videos/480p15/partial_movie_ files/BookmarkExample/uncached _00001.mp4', '/Users/bantje/manim-test/medi a/videos/480p15/partial_movie_ files/BookmarkExample/uncached _00002.mp4', '/Users/bantje/manim-test/medi a/videos/480p15/partial_movie_ files/BookmarkExample/uncached _00003.mp4', '/Users/bantje/manim-test/medi a/videos/480p15/partial_movie_ files/BookmarkExample/uncached _00004.mp4'] [05/12/23 12:21:44] INFO scene_file_writer.py:736 File ready at '/Users/bantje/manim-test/medi a/videos/480p15/BookmarkExampl e.mp4' INFO Subcaption file has been scene_file_writer.py:731 written as /Users/bantje/manim-test/media /videos/480p15/BookmarkExample .srt INFO Rendered BookmarkExample scene.py:241 Played 26 animations R ```System specifications
System Details
- macOS 10.15 (Catalina) - RAM: 32 GB - Python 3.11 - Installed modules (provide output from `pip list`): ``` Package Version ------------------------------------ -------- appnope 0.1.3 asttokens 2.2.1 azure-cognitiveservices-speech 1.28.0 backcall 0.2.0 certifi 2023.5.7 charset-normalizer 3.1.0 click 8.1.3 click-default-group 1.2.2 cloup 0.13.1 colour 0.1.5 comm 0.1.3 Cython 0.29.34 debugpy 1.6.7 decorator 5.1.1 executing 1.2.0 ffmpeg-python 0.2.0 filelock 3.12.0 fsspec 2023.5.0 future 0.18.3 glcontext 2.3.7 gTTS 2.3.2 huggingface-hub 0.14.1 humanhash3 0.0.6 idna 3.4 ipykernel 6.23.0 ipython 8.12.2 isosurfaces 0.1.0 jedi 0.18.2 Jinja2 3.1.2 jupyter_client 8.2.0 jupyter_core 5.3.0 manim 0.17.3 manim-voiceover 0.3.0 ManimPango 0.4.3 mapbox-earcut 1.0.1 markdown-it-py 2.2.0 MarkupSafe 2.1.2 matplotlib-inline 0.1.6 mdurl 0.1.2 moderngl 5.8.2 moderngl-window 2.4.3 more-itertools 9.1.0 mpmath 1.3.0 multipledispatch 0.6.0 mutagen 1.46.0 nest-asyncio 1.5.6 networkx 2.8.8 numpy 1.24.3 openai-whisper 20230117 packaging 23.1 parso 0.8.3 pexpect 4.8.0 pickleshare 0.7.5 Pillow 9.5.0 pip 23.1.2 platformdirs 3.5.0 prompt-toolkit 3.0.38 psutil 5.9.5 ptyprocess 0.7.0 pure-eval 0.2.2 PyAudio 0.2.13 pycairo 1.23.0 pydub 0.25.1 pyglet 2.0.7 Pygments 2.15.1 pynput 1.7.6 pyobjc-core 9.1.1 pyobjc-framework-ApplicationServices 9.1.1 pyobjc-framework-Cocoa 9.1.1 pyobjc-framework-Quartz 9.1.1 pyrr 0.10.3 python-dateutil 2.8.2 python-dotenv 0.21.1 PyYAML 6.0 pyzmq 25.0.2 regex 2023.5.5 requests 2.30.0 rich 13.3.5 scipy 1.10.1 screeninfo 0.8.1 setuptools 67.6.1 six 1.16.0 skia-pathops 0.7.4 sox 1.4.1 srt 3.5.3 stable-ts 1.1.0 stack-data 0.6.2 svgelements 1.9.3 sympy 1.12 tokenizers 0.13.3 torch 2.0.1 torchaudio 2.0.2 tornado 6.3.1 tqdm 4.65.0 traitlets 5.9.0 transformers 4.29.0 typing_extensions 4.5.0 urllib3 2.0.2 watchdog 2.3.1 wcwidth 0.2.6 wheel 0.40.0 ```LaTeX details
+ LaTeX distribution: TeX Live 2023FFMPEG
Output of `ffmpeg -version`: ``` ffmpeg version 6.0 Copyright (c) 2000-2023 the FFmpeg developers built with Apple clang version 14.0.3 (clang-1403.0.22.14.1) configuration: --prefix=/usr/local/Cellar/ffmpeg/6.0 --enable-shared --enable-pthreads --enable-version3 --cc=clang --host-cflags= --host-ldflags= --enable-ffplay --enable-gnutls --enable-gpl --enable-libaom --enable-libaribb24 --enable-libbluray --enable-libdav1d --enable-libmp3lame --enable-libopus --enable-librav1e --enable-librist --enable-librubberband --enable-libsnappy --enable-libsrt --enable-libsvtav1 --enable-libtesseract --enable-libtheora --enable-libvidstab --enable-libvmaf --enable-libvorbis --enable-libvpx --enable-libwebp --enable-libx264 --enable-libx265 --enable-libxml2 --enable-libxvid --enable-lzma --enable-libfontconfig --enable-libfreetype --enable-frei0r --enable-libass --enable-libopencore-amrnb --enable-libopencore-amrwb --enable-libopenjpeg --enable-libspeex --enable-libsoxr --enable-libzmq --enable-libzimg --disable-libjack --disable-indev=jack --enable-videotoolbox libavutil 58. 2.100 / 58. 2.100 libavcodec 60. 3.100 / 60. 3.100 libavformat 60. 3.100 / 60. 3.100 libavdevice 60. 1.100 / 60. 1.100 libavfilter 9. 3.100 / 9. 3.100 libswscale 7. 1.100 / 7. 1.100 libswresample 4. 10.100 / 4. 10.100 libpostproc 57. 1.100 / 57. 1.100 ```Additional comments
This behaviour hints at inconsistencies on the Azure side of things or an issue in parsing its output.
I would also be interested in possible workarounds:
whisper
to override the word boundaries provided by Azure(?) – addingtranscription_model='base'
had no effect though.