OpenVoiceOS / ovos-tts-plugin-piper

Apache License 2.0
20 stars 11 forks source link

sentences randomly fail to synth #31

Open JarbasAl opened 1 month ago

JarbasAl commented 1 month ago

while benchmarking plugins in https://github.com/TigreGotico/tts-benchmarks

i noticed i am unable to benchmark piper plugin because some sentences randomly fail, worse, sometimes this crashes the whole python process with some models

failed synths are logged in the benchmark json files under "failed_synths", example https://github.com/TigreGotico/tts-benchmarks/blob/master/benchmark_tts_es.json#L762

        "failed_synths": [
            "Big sharks swimming faster grab cold fish now.",
            "Six big devils from Japan quickly forgot how to waltz.",
            "Just keep examining every low bid quoted for zinc etchings.",
            "All questions asked by five watched experts amaze the judge.",
            "Amazingly few discotheques provide jukeboxes.",
            "The jay, pig, fox, zebra, and my wolves quack!",
            "Vampires kill zombies by quacking when we jog near fog.",
            "Twelve big black bugs bleed blue blood.",
            "Jack quickly faxed his extra prize-winning zebra jugs.",
            "Fred, the vampire, zapped a quick jabberwock.",
            "Glowing fish quickly zipped past the brave jaguar’s mouth.",
            "A giant box of jugs seized my waxed frogs quickly.",
            "Five brave men juggle swords by the hazy fog.",
            "Mix jumping zebras with brave tigers in quiet fog.",
            "A hefty jabberwock quickly vexes five big men.",
            "Fizzled quail jabs waxy box of bright frogs.",
            "A dozen jumpy frogs vex the wizard’s big quiz.",
            "Zany wizards juggle five bright boxes over quick fog.",
            "Victor juggled five bright zigzagged boxes quickly in the fog."
        ]
        "failed_synths": [
            "En otoño, las hojas caen lentamente de los árboles.",
            "Los niños juegan en la playa mientras el sol se pone.",
            "Hoy el sol brilla intensamente y el cielo está azul.",
            "Los pájaros cantan fuerte a la luz de la mañana.",
            "Pablo pasea por el bosque y disfruta del aire fresco.",
            "El viejo despertador sonó a las siete en punto.",
            "Cinco patos nadan pacíficamente en el estanque tranquilo.",
            "El coche circula rápidamente por la carretera sinuosa.",
            "El chico corrió rápido para alcanzar el autobús.",
            "El gatito ronronea suavemente sobre el cojín.",
            "El viento sopla fuerte entre los altos árboles.",
            "La campana sonó doce veces al mediodía.",
            "Los niños estaban felices de ver la primera nieve del año.",
            "El pan estaba fresco y crujiente cuando salió del horno.",
            "El caballo galopa por la amplia pradera verde.",
            "El pájaro cantaba alto en el árbol junto a la casa.",
            "La pequeña barca se balanceaba sobre las olas del lago.",
            "El viejo hombre contaba historias de su juventud.",
            "El gato saltó elegantemente por la ventana.",
            "El avión aterrizó con seguridad a pesar del fuerte viento.",
            "Las hojas de los árboles susurraban en la suave brisa.",
            "La familia hizo un picnic a la sombra del gran roble.",
            "El reloj en la pared hacía tic-tac suavemente de fondo.",
            "El sol se ponía lentamente detrás de las montañas.",
            "El niño recogía flores de colores en el campo."
        ]
JarbasAl commented 1 month ago

error logs for alan-low

Generating TTS for <ovos_tts_plugin_piper.PiperTTSPlugin object at 0x78c88b916e90>/en/alan-low:  14%|█▍        | 7/49 [00:00<00:05,  7.45sentence/s]
Traceback (most recent call last):
  File "/home/miro/PycharmProjects/tts-benchmarks/benchmark_rtf_en.py", line 108, in <module>
    rtf, wavs, failed = get_rtf(sentences=sentences, lang=lang, plug=tts, voice=voice)
                        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/miro/PycharmProjects/tts-benchmarks/benchmark_rtf_en.py", line 36, in get_rtf
    plug.get_tts(s, wav_file=wav_path, lang=lang, voice=voice)
  File "/home/miro/PycharmProjects/ovos-core/.venv/lib/python3.11/site-packages/ovos_tts_plugin_piper/__init__.py", line 386, in get_tts
    engine.synthesize(sentence, f,
  File "/home/miro/PycharmProjects/ovos-core/.venv/lib/python3.11/site-packages/piper/voice.py", line 95, in synthesize
    for audio_bytes in self.synthesize_stream_raw(
  File "/home/miro/PycharmProjects/ovos-core/.venv/lib/python3.11/site-packages/piper/voice.py", line 123, in synthesize_stream_raw
    yield self.synthesize_ids_to_raw(
          ^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/miro/PycharmProjects/ovos-core/.venv/lib/python3.11/site-packages/ovos_tts_plugin_piper/__init__.py", line 77, in synthesize_ids_to_raw
    audio = self.session.run(None, args, )[0].squeeze((0, 1))
            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/miro/PycharmProjects/ovos-core/.venv/lib/python3.11/site-packages/onnxruntime/capi/onnxruntime_inference_collection.py", line 220, in run
    return self._sess.run(output_names, input_feed, run_options)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
onnxruntime.capi.onnxruntime_pybind11_state.RuntimeException: [ONNXRuntimeError] : 6 : RUNTIME_EXCEPTION : Non-zero status code returned while running Reshape node. Name:'/Reshape_1' Status Message: /onnxruntime_src/onnxruntime/core/providers/cpu/tensor/reshape_helper.h:28 onnxruntime::ReshapeHelper::ReshapeHelper(const onnxruntime::TensorShape&, onnxruntime::TensorShapeVector&, bool) i < input_shape.NumDimensions() was false. The dimension with value zero exceeds the dimension size of the input tensor.

ERROR with sentence 'Big sharks swimming faster grab cold fish now.' : [ONNXRuntimeError] : 6 : RUNTIME_EXCEPTION : Non-zero status code returned while running Reshape node. Name:'/Reshape_1' Status Message: /onnxruntime_src/onnxruntime/core/providers/cpu/tensor/reshape_helper.h:28 onnxruntime::ReshapeHelper::ReshapeHelper(const onnxruntime::TensorShape&, onnxruntime::TensorShapeVector&, bool) i < input_shape.NumDimensions() was false. The dimension with value zero exceeds the dimension size of the input tensor.