Open CStrue opened 1 month ago
I have the same problem (but with a slightly different error) and reported it also here: https://github.com/artibex/piper-http/issues/3
@CStrue just in case, i fixed it by using this:
pip install -U piper-phonemize pip install onnxruntime==1.17.1
instead of using the requirements.txt. Current version 1.18 doesnt seem to work
I found a solution to it tonight. I'm going to paste it later. Basicall it seems to be a problem in the state management and loading everything within the get/post method of the server (inluding the voice models) solves the problem since it becomes stateless this way.
server side code: I moved all the code into the flask decorator method. It seems necesessary to do the voice loading every time to avoid the issue. You should be able to move the code also in the original http python file. The con is that the voice loading takes some time, but for my purpose its still fast enough.
from flask import Flask, request, Response
import io
import wave
import logging
from piper import PiperVoice
app = Flask(__name__)
logging.basicConfig(level=logging.ERROR)
@app.route('/synthesize', methods=['GET'])
def synthesize():
try:
text = request.args.get('text', '')
if not text:
return "Text parameter is required", 400
voice = PiperVoice.load(
'voices/german/de_DE-thorsten-high.onnx',
config_path='voices/german/de_DE-thorsten-high.onnx.json'
)
wav_io = io.BytesIO()
with wave.open(wav_io, 'wb') as wav_file:
voice.synthesize(text, wav_file)
wav_io.seek(0)
return Response(
wav_io,
mimetype='audio/wav',
headers={"Content-Disposition": "attachment;filename=output.wav"}
)
except Exception as e:
logging.error(f"Error during synthesis: {e}")
return "An error occurred during synthesis", 500
if __name__ == "__main__":
app.run(host='0.0.0.0', port=5000)
python client side code to test the server:
import requests
import sounddevice as sd
import numpy as np
import io
import wave
textToSpeak = "Mein Name ist Skynet und ich bin dein persönlicher Assistent. Wie kann ich dir heute helfen?."
urlPiper = "http://localhost:5000/synthesize"
payload = {'text': textToSpeak}
response = requests.get(urlPiper, params=payload, stream=True)
if response.status_code == 200:
with io.BytesIO() as audio_stream:
for chunk in response.iter_content(chunk_size=128):
audio_stream.write(chunk)
audio_stream.seek(0)
with wave.open(audio_stream, 'rb') as wf:
samplerate = wf.getframerate()
channels = wf.getnchannels()
width = wf.getsampwidth()
data = wf.readframes(wf.getnframes())
if width == 2: # 16-bit samples
dtype = np.int16
elif width == 4: # 32-bit samples
dtype = np.int32
else:
raise ValueError(f"Unsupported sample width: {width}")
audio_data = np.frombuffer(data, dtype=dtype)
if channels == 2:
audio_data = audio_data.reshape(-1, 2)
print("Available devices:")
for i, device in enumerate(sd.query_devices()):
print(f"{i}: {device['name']}")
device_id = 11
print(f"Using device ID: {device_id}")
sd.play(audio_data, samplerate=samplerate, device=device_id)
sd.wait()
else:
print(f"Failed to get audio. Status code: {response.status_code}")
print("Response content:", response.text)
~/projects/piper/piper/src/python_run $ python3 -m piper.http_server --model ../../../voices/german/de_DE-thorsten-high.onnx
...
INFO:werkzeug:Press CTRL+C to quit
=================== First call - successful 200
INFO:werkzeug: - - [23/Jul/2024 22:26:24] "GET /?text=Mein+Name+ist+Skynet+und+ich+bin+dein+persönlicher+Assistent.+Wie+kann+ich+dir+heute+helfen?. HTTP/1.1" 200 -
===================== second call - throwing error
2024-07-23 22:26:29.666429175 [E:onnxruntime:, sequential_executor.cc:516 ExecuteKernel] Non-zero status code returned while running GatherElements node. Name:'/dp/flows.7/GatherElements_3' Status Message: /onnxruntime_src/onnxruntime/core/providers/cpu/tensor/gather_elements.cc:154 void onnxruntime::core_impl(const Tensor, const Tensor, Tensor, int64_t, concurrency::ThreadPool) [with Tin = long int; int64_t = long int] GatherElements op: Out of range value in index tensor
ERROR:http_server:Exception on / [GET] Traceback (most recent call last): File "/home/admin/projects/piper/.venv/lib/python3.11/site-packages/flask/app.py", line 1473, in wsgi_app response = self.full_dispatch_request() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/admin/projects/piper/.venv/lib/python3.11/site-packages/flask/app.py", line 882, in full_dispatch_request rv = self.handle_user_exception(e) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/admin/projects/piper/.venv/lib/python3.11/site-packages/flask/app.py", line 880, in full_dispatch_request rv = self.dispatch_request() ^^^^^^^^^^^^^^^^^^^^^^^ File "/home/admin/projects/piper/.venv/lib/python3.11/site-packages/flask/app.py", line 865, in dispatch_request return self.ensure_sync(self.view_functions[rule.endpoint])(view_args) # type: ignore[no-any-return] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/admin/projects/piper/piper/src/python_run/piper/http_server.py", line 119, in app_synthesize voice.synthesize(text, wav_file, synthesize_args) File "/home/admin/projects/piper/piper/src/python_run/piper/voice.py", line 104, in synthesize for audio_bytes in self.synthesize_stream_raw( File "/home/admin/projects/piper/piper/src/python_run/piper/voice.py", line 132, in synthesize_stream_raw yield self.synthesize_ids_to_raw( ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/admin/projects/piper/piper/src/python_run/piper/voice.py", line 183, in synthesize_ids_to_raw audio = self.session.run(None, args, )[0].squeeze((0, 1)) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/admin/projects/piper/.venv/lib/python3.11/site-packages/onnxruntime/capi/onnxruntime_inference_collection.py", line 220, in run return self._sess.run(output_names, input_feed, run_options) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ onnxruntime.capi.onnxruntime_pybind11_state.RuntimeException: [ONNXRuntimeError] : 6 : RUNTIME_EXCEPTION : Non-zero status code returned while running GatherElements node. Name:'/dp/flows.7/GatherElements_3' Status Message: /onnxruntime_src/onnxruntime/core/providers/cpu/tensor/gather_elements.cc:154 void onnxruntime::core_impl(const Tensor, const Tensor, Tensor, int64_t, concurrency::ThreadPool) [with Tin = long int; int64_t = long int] GatherElements op: Out of range value in index tensor
INFO:werkzeug: - - [23/Jul/2024 22:26:29] "GET /?text=Mein+Name+ist+Skynet+und+ich+bin+dein+persönlicher+Assistent.+Wie+kann+ich+dir+heute+helfen?. HTTP/1.1" 500 -