search

8090-inc / xrx-sample-apps

Sample web apps built with xRx-Core
https://8090-inc.github.io/xrx-core/
Apache License 2.0
92 stars 14 forks source link

Patient App - TTS Issue #14

Open monkeydust opened 4 days ago

monkeydust commented 4 days ago

Tried using Groq (default) TTS - using same API key for LLM_API_KEY and GROQ_SST_API_KEY but keep getting error below.

I tried changing STT_PROVIDER = "deepgram" and set DG_API_KEY but it always uses Groq and throws same error.

rx-orchestrator | initial response sent to STT 2024-09-24 09:35:49 xrx-orchestrator | [08:35:48 UTC] INFO: incoming request 2024-09-24 09:35:49 xrx-orchestrator | reqId: "req-1" 2024-09-24 09:35:49 xrx-orchestrator | req: { 2024-09-24 09:35:49 xrx-orchestrator | "method": "GET", 2024-09-24 09:35:49 xrx-orchestrator | "url": "/api/v1/ws", 2024-09-24 09:35:49 xrx-orchestrator | "hostname": "localhost:8000", 2024-09-24 09:35:49 xrx-orchestrator | "remoteAddress": "172.18.0.1", 2024-09-24 09:35:49 xrx-orchestrator | "remotePort": 35752 2024-09-24 09:35:49 xrx-orchestrator | } 2024-09-24 09:35:49 xrx-orchestrator | [08:35:48 UTC] DEBUG: preValidation hook 2024-09-24 09:35:49 xrx-orchestrator | [08:35:48 UTC] DEBUG: Connection opened 2024-09-24 09:35:49 xrx-orchestrator | [08:35:48 UTC] DEBUG: {"host":"localhost:8000","connection":"Upgrade","pragma":"no-cache","cache-control":"no-cache","user-agent":"Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/129.0.0.0 Safari/537.36 Edg/129.0.0.0","upgrade":"websocket","origin":"http://localhost:3000","sec-websocket-version":"13","accept-encoding":"gzip, deflate, br, zstd","accept-language":"en-GB,en;q=0.9,en-US;q=0.8","sec-websocket-key":"O5SiDIMkdDvR7cKJQLuidg==","sec-websocket-extensions":"permessage-deflate; client_max_window_bits"} 2024-09-24 09:35:49 xrx-orchestrator | [08:35:48 UTC] DEBUG: STT Opening WebSocket:ws://xrx-stt:8001/api/v1/ws 2024-09-24 09:35:49 xrx-orchestrator | [08:35:48 UTC] DEBUG: TTS Opening WebSocket:ws://xrx-tts:8002/api/v1/ws 2024-09-24 09:35:49 xrx-orchestrator | [08:35:48 UTC] DEBUG: STT WebSocket opened 2024-09-24 09:35:49 xrx-orchestrator | [08:35:48 UTC] DEBUG: TTS WebSocket opened 2024-09-24 09:35:51 xrx-client | ⚠ The "images.domains" configuration is deprecated. Please use "images.remotePatterns" configuration instead. 2024-09-24 09:35:51 xrx-client | ⨯ Error: 'sharp' is required to be installed in standalone mode for the image optimization to function correctly. Read more at: https://nextjs.org/docs/messages/sharp-missing-in-production 2024-09-24 09:35:51 xrx-client | ⨯ Failed to write image to cache 8bP9JR6UNW0uOjfukrlwvbAvrsrDZUbVnRb-NJWyjWk= [Error: EACCES: permission denied, mkdir '/nextjs-client/.next/cache'] { 2024-09-24 09:35:51 xrx-client | errno: -13, 2024-09-24 09:35:51 xrx-client | code: 'EACCES', 2024-09-24 09:35:51 xrx-client | syscall: 'mkdir', 2024-09-24 09:35:51 xrx-client | path: '/nextjs-client/.next/cache' 2024-09-24 09:35:51 xrx-client | } 2024-09-24 09:35:57 xrx-stt | INFO:groq_stt:Transcribing audio using Groq API... 2024-09-24 09:35:57 xrx-stt | INFO:groq_stt:Groq model initialized. 2024-09-24 09:35:58 xrx-stt | INFO:httpx:HTTP Request: POST https://api.groq.com/openai/v1/audio/transcriptions "HTTP/1.1 401 Unauthorized" 2024-09-24 09:35:58 xrx-stt | ERROR: Exception in ASGI application 2024-09-24 09:35:58 xrx-stt | Traceback (most recent call last): 2024-09-24 09:35:58 xrx-stt | File "/usr/local/lib/python3.10/site-packages/uvicorn/protocols/websockets/websockets_impl.py", line 244, in run_asgi 2024-09-24 09:35:58 xrx-stt | result = await self.app(self.scope, self.asgi_receive, self.asgi_send) # type: ignore[func-returns-value] 2024-09-24 09:35:58 xrx-stt | File "/usr/local/lib/python3.10/site-packages/uvicorn/middleware/proxy_headers.py", line 70, in call 2024-09-24 09:35:58 xrx-stt | return await self.app(scope, receive, send) 2024-09-24 09:35:58 xrx-stt | File "/usr/local/lib/python3.10/site-packages/fastapi/applications.py", line 1054, in call 2024-09-24 09:35:58 xrx-stt | await super().call(scope, receive, send) 2024-09-24 09:35:58 xrx-stt | File "/usr/local/lib/python3.10/site-packages/starlette/applications.py", line 123, in call 2024-09-24 09:35:58 xrx-stt | await self.middleware_stack(scope, receive, send) 2024-09-24 09:35:58 xrx-stt | File "/usr/local/lib/python3.10/site-packages/starlette/middleware/errors.py", line 151, in call 2024-09-24 09:35:58 xrx-stt | await self.app(scope, receive, send) 2024-09-24 09:35:58 xrx-stt | File "/usr/local/lib/python3.10/site-packages/starlette/middleware/exceptions.py", line 65, in call 2024-09-24 09:35:58 xrx-stt | await wrap_app_handling_exceptions(self.app, conn)(scope, receive, send) 2024-09-24 09:35:58 xrx-stt | File "/usr/local/lib/python3.10/site-packages/starlette/_exception_handler.py", line 64, in wrapped_app 2024-09-24 09:35:58 xrx-stt | raise exc 2024-09-24 09:35:58 xrx-stt | File "/usr/local/lib/python3.10/site-packages/starlette/_exception_handler.py", line 53, in wrapped_app 2024-09-24 09:35:58 xrx-stt | await app(scope, receive, sender) 2024-09-24 09:35:58 xrx-stt | File "/usr/local/lib/python3.10/site-packages/starlette/routing.py", line 756, in call 2024-09-24 09:35:58 xrx-stt | await self.middleware_stack(scope, receive, send) 2024-09-24 09:35:58 xrx-stt | File "/usr/local/lib/python3.10/site-packages/starlette/routing.py", line 776, in app 2024-09-24 09:35:58 xrx-stt | await route.handle(scope, receive, send) 2024-09-24 09:35:58 xrx-stt | File "/usr/local/lib/python3.10/site-packages/starlette/routing.py", line 373, in handle 2024-09-24 09:35:58 xrx-stt | await self.app(scope, receive, send) 2024-09-24 09:35:58 xrx-stt | File "/usr/local/lib/python3.10/site-packages/starlette/routing.py", line 96, in app 2024-09-24 09:35:58 xrx-stt | await wrap_app_handling_exceptions(app, session)(scope, receive, send) 2024-09-24 09:35:58 xrx-stt | File "/usr/local/lib/python3.10/site-packages/starlette/_exception_handler.py", line 64, in wrapped_app 2024-09-24 09:35:58 xrx-stt | raise exc 2024-09-24 09:35:58 xrx-stt | File "/usr/local/lib/python3.10/site-packages/starlette/_exception_handler.py", line 53, in wrapped_app 2024-09-24 09:35:58 xrx-stt | await app(scope, receive, sender) 2024-09-24 09:35:58 xrx-stt | File "/usr/local/lib/python3.10/site-packages/starlette/routing.py", line 94, in app 2024-09-24 09:35:58 xrx-stt | await func(session) 2024-09-24 09:35:58 xrx-stt | File "/usr/local/lib/python3.10/site-packages/fastapi/routing.py", line 348, in app 2024-09-24 09:35:58 xrx-stt | await dependant.call(**values) 2024-09-24 09:35:58 xrx-stt | File "/stt/main.py", line 45, in websocket_endpoint 2024-09-24 09:35:58 xrx-stt | result = await stt_model.transcribe(data) 2024-09-24 09:35:58 xrx-stt | File "/stt/groq_stt.py", line 72, in transcribe 2024-09-24 09:35:58 xrx-stt | transcription = model.audio.transcriptions.create( 2024-09-24 09:35:58 xrx-stt | File "/usr/local/lib/python3.10/site-packages/groq/resources/audio/transcriptions.py", line 118, in create 2024-09-24 09:35:58 xrx-stt | return self._post( 2024-09-24 09:35:58 xrx-stt | File "/usr/local/lib/python3.10/site-packages/groq/_base_client.py", line 1225, in post 2024-09-24 09:35:58 xrx-stt | return cast(ResponseT, self.request(cast_to, opts, stream=stream, stream_cls=stream_cls)) 2024-09-24 09:35:58 xrx-stt | File "/usr/local/lib/python3.10/site-packages/groq/_base_client.py", line 920, in request 2024-09-24 09:35:58 xrx-stt | return self._request( 2024-09-24 09:35:58 xrx-stt | File "/usr/local/lib/python3.10/site-packages/groq/_base_client.py", line 1018, in _request 2024-09-24 09:35:58 xrx-stt | raise self._make_status_error_from_response(err.response) from None 2024-09-24 09:35:58 xrx-stt | groq.AuthenticationError: Error code: 401 - {'error': {'message': 'Invalid API Key', 'type': 'invalid_request_error', 'code': 'invalid_api_key'}} 2024-09-24 09:35:58 xrx-stt | INFO: connection closed

chrislott commented 3 days ago

Thanks monkeydust, we'll look at this today.

chrislott commented 3 days ago

@monkeydust , can you look in your .env file to see if there is a duplicate DG_API_KEY under the TTS section? That is probably overwriting the Deepgram API key, which is why you're seeing a 401. We will fix this by leaving it commented out by default.

monkeydust commented 3 days ago

Its referenced twice so I put in the key twice, either way it never pointed to Deepgram, per log trace it keeps using Groq and I cant understand why thats throwing an error. I tried re-creating the key as well.

chrislott commented 3 days ago

Were you able to get it working with a Groq API key?

alessandro-neri commented 3 days ago

Hi @monkeydust, thank you for reporting the issue. Could you please confirm that you are changing the STT module and not the TTS module?

There should be 2 .env variables: TTS_PROVIDER="elevenlabs" and STT_PROVIDER="groq". Please ensure that you change the model in the appropriate variable.

monkeydust commented 3 days ago

Hi @alessandro-neri below is the .env redacted - I gave up with Groq so created a Deepgram account but as I said it never seems to initiate Deepgram and still tries to do the TTS using Groq. 


# ============== Configuration ==============
# Minimal configuration to get started.
# Get your API keys from the following URLs:
# - Groq: https://console.groq.com/keys/
# - ElevenLabs: https://elevenlabs.io/app/speech-synthesis/text-to-speech
# =============================================

# === LLM Configuration ===
LLM_API_KEY=<redacted>
LLM_BASE_URL="https://api.groq.com/openai/v1"
LLM_MODEL_ID="llama3-70b-8192"

# === Speech-to-Text (STT) Configuration ===
STT_PROVIDER="deepgram"     # Alternative providers are  "deepgram", "faster_whisper"
#GROQ_STT_API_KEY=<redacted>
#GROQ_STT_API_KEY=<redacted>

# === Text-to-Speech (TTS) Configuration ===
TTS_PROVIDER="elevenlabs"       # Alternative providers: "deepgram", "openai", "cartesia"
ELEVENLABS_API_KEY=<redacted>
ELEVENLABS_VOICE_ID="your_elevenlabs_voice_id"

# =============================================
# Customize your configuration below
# =============================================

# === Reasoning Configuration ===
INITIAL_RESPONSE="Hello! How can I help you?"

# === Speech-to-Text (STT) Configuration ===
#DG_API_KEY=  # required if you want to use Deepgram
# STT_SAMPLE_RATE="16000"

# === Text-to-Speech (TTS) Configuration ===
# Get your API keys from the following URLs:
# - Deepgram: https://deepgram.com/
# - OpenAI: https://openai.com/
TTS_SAMPLE_RATE="24000"
# Deepgram configuration
DG_TTS_MODEL_VOICE="aura-asteria-en"
DG_API_KEY=<redacted>
# OpenAI configuration
OPENAI_API_KEY=<redacted>
OPENAI_TTS_MODEL="tts-1"
OPENAI_TTS_VOICE="alloy"
# CACHE_DIR="your_cache_directory_path"

# === Guardrails Proxy Configuration ===
# AGENT_HOST="xrx-guardrails"
# AGENT_PORT="8004"
# GUARDRAILS_AGENT_HOST="xrx-reasoning"
# GUARDRAILS_AGENT_PORT="8003"

# === LLM Observability Configuration ===
LLM_OBSERVABILITY_LIBRARY="none"
# Alternatives: 
# LLM_OBSERVABILITY_LIBRARY="langsmith"
# LLM_OBSERVABILITY_LIBRARY="langfuse"

# Langfuse configuration
LANGFUSE_SECRET_KEY="your_langfuse_secret_key"
LANGFUSE_PUBLIC_KEY="your_langfuse_public_key"
LANGFUSE_HOST="http://host.docker.internal:3001"
# For localhost: LANGFUSE_HOST="http://localhost:3001"
# For cloud-hosted Langfuse: LANGFUSE_HOST="https://us.cloud.langfuse.com"

# LangSmith configuration
LANGCHAIN_TRACING_V2="true"
LANGCHAIN_API_KEY="your_langsmith_api_key"
LANGCHAIN_PROJECT="your_langsmith_project_name"
LANGCHAIN_ENDPOINT="https://api.smith.langchain.com"

# === UI ===
NEXT_PUBLIC_UI_DEBUG_MODE="false"

# === Redis Configuration ===
REDIS_HOST="localhost"

# === LLM Configuration ===
# JSON fixing model (if needed)
LLM_MODEL_ID_JSON_FIXER="llama3-70b-8192"

# === Orchestrator Configuration ===
# AGENT_WAIT_MS="your_agent_wait_time"
# SAMPLE_RATE="your_sample_rate"
# STT_WAIT_MS="your_stt_wait_time"

# STT_HOST="xrx-stt"
# STT_PORT="8001"
# STT_PATH="/api/v1/ws"

# TTS_HOST="xrx-tts"
# TTS_PORT="8002"
# TTS_PATH="/api/v1/ws"

# AGENT_PATH="/run-reasoning-agent"
alessandro-neri commented 3 days ago

Hi @monkeydust

I've tried using your .env file by replacing the API keys with mine, and it works perfectly with Deepgram. Are you running the app using Docker? If so, you should also update REDIS_HOST="xrx-redis" as indicated in the example env file. We strongly advise running the app in a Docker container rather than deploying each part locally. Let me know if the error persists. If it does, I will wait for more information to reproduce the error.

chrislott commented 3 days ago

Also, please try switching to the develop branch. There is some fixes for the patient intake demo.

monkeydust commented 3 days ago

@chrislott hi getting a bit further but not there yet...

  1. On windows 11 and yes using docker container.

  2. After cloning dev branch, using same keys and setting REDIS_HOST="xrx-redis" I run this:

C:\Users\admin\xrx-sample-apps\patient-information-app>docker-compose up --build

  1. Then I am using windows docker desktop to manage the container (start/stop)

  2. The app does load but on localhost:8000 where it asks me for questions. When I answer it does seem to transcribe now on Groq but throws error (at very bottom ).

  3. If I run localhost:8003 I get this error:

    { "detail": "Not Found" }

  4. If I run per the readme the command given to launch container I get this error:

C:\Users\admin\xrx-sample-apps\patient-information-app>docker run -p 8003:8003 --env-file .env patient-information-agent Unable to find image 'patient-information-agent:latest' locally docker: Error response from daemon: pull access denied for patient-information-agent, repository does not exist or may require 'docker login': denied: requested access to the resource is denied. See 'docker run --help'.

Hopefully this helps diagnose what the issue is. Thanks

**

Error: 991446, 'compression_ratio': 0.82857144, 'no_speech_prob': 0.10233396}], x_groq={'id': 'req_01j8jqtcscek2sbnts97t9z78q'}) 2024-09-24 20:23:36 xrx-tts | INFO:main:Received action: cancel 2024-09-24 20:23:36 xrx-tts | INFO:main:Received cancel action 2024-09-24 20:23:36 xrx-reasoning | 2024-09-24 19:23:36,767 INFO:Received request body: {'messages': [{'role': 'user', 'content': 'Yes, go ahead.'}, {'role': 'user', 'content': 'Hello.'}, {'role': 'user', 'content': 'Yes, can you?'}, {'role': 'user', 'content': 'You count.'}, {'role': 'user', 'content': 'questions.'}, {'role': 'user', 'content': 'please ask me some questions.'}], 'session': {'guid': '76443260-3726-45c2-b564-361179f4d37f'}, 'action': {}} 2024-09-24 20:23:36 xrx-reasoning | 2024-09-24 19:23:36,767 INFO:Request headers: Headers({'host': 'xrx-reasoning:8003', 'connection': 'keep-alive', 'content-type': 'application/json', 'accept': 'text/event-stream', 'accept-language': '*', 'sec-fetch-mode': 'cors', 'user-agent': 'node', 'accept-encoding': 'gzip, deflate', 'content-length': '328'}) 2024-09-24 20:23:36 xrx-reasoning | 2024-09-24 19:23:36,767 INFO:Request client: Address(host='172.18.0.7', port=36068) 2024-09-24 20:23:36 xrx-reasoning | 2024-09-24 19:23:36,767 INFO:Created task with task ID: 534c591b-3b75-4382-87a5-eae27cc908cd 2024-09-24 20:23:36 xrx-reasoning | 2024-09-24 19:23:36,767 ERROR:An error occurred while processing the request: Error 111 connecting to localhost:6379. 111. 2024-09-24 20:23:36 xrx-reasoning | INFO: 172.18.0.7:36068 - "POST /run-reasoning-agent HTTP/1.1" 500 Internal Server Error 2024-09-24 20:23:37 xrx-orchestrator | [19:23:36 UTC] DEBUG: Received from STT: please ask me some questions. 2024-09-24 20:23:37 xrx-orchestrator | [19:23:36 UTC] DEBUG: Received from STT: please ask me some questions. 2024-09-24 20:23:37 xrx-orchestrator | [19:23:36 UTC] DEBUG: Cancelling all agent activity 2024-09-24 20:23:37 xrx-orchestrator | [19:23:36 UTC] DEBUG: Sending to agent:please ask me some questions. 2024-09-24 20:23:37 xrx-orchestrator | [19:23:36 UTC] DEBUG: send chat history to agent 2024-09-24 20:23:37 xrx-orchestrator | [19:23:36 UTC] DEBUG: Sending to agent xrx-reasoning:8003/run-reasoning-agent: [{"role":"user","content":"Yes, go ahead."},{"role":"user","content":"Hello."},{"role":"user","content":"Yes, can you?"},{"role":"user","content":"You count."},{"role":"user","content":"questions."},{"role":"user","content":"please ask me some questions."}] 2024-09-24 20:23:37 xrx-orchestrator | [19:23:36 UTC] DEBUG: Sending to agent: {"guid":"76443260-3726-45c2-b564-361179f4d37f"} 2024-09-24 20:23:37 xrx-orchestrator | [19:23:36 UTC] DEBUG: Raw output: {"detail":"Error 111 connecting to localhost:6379. 111."}

chrislott commented 3 days ago

We just updated main -- can you do a fresh clone and copy your env file into the patient app and try docker compose up again?

monkeydust commented 2 days ago

@chrislott painfully close now!

So its asks me for questions, I can see its transcribing my voice but its not asking the questions (no sound) looking at logs its not resolving the voice_id for elevenlabs, I have tried the actual voice_id and name neither works.

E.g.

voice_id = CwhRBWXzGAHq8TQ4Fs17 for Roger

from https://api.elevenlabs.io/v1/voices 

2024-09-25 18:12:08 xrx-orchestrator  | [17:12:07 UTC] DEBUG: Received from STT:  What are your questions?
2024-09-25 18:12:08 xrx-orchestrator  | [17:12:07 UTC] DEBUG: Cancelling all agent activity
2024-09-25 18:12:08 xrx-orchestrator  | [17:12:07 UTC] DEBUG: Sending to agent:What are your questions?
2024-09-25 18:12:08 xrx-orchestrator  | [17:12:07 UTC] DEBUG: send chat history to agent
2024-09-25 18:12:08 xrx-orchestrator  | [17:12:07 UTC] DEBUG: Sending to agent xrx-reasoning:8003/run-reasoning-agent: [{"role":"user","content":"Yes, go for it."},{"role":"assistant","content":"{\n   \"information-received\": {},\n   \"response\": \"Can you please confirm your first and last name?\"\n}"},{"role":"user","content":"What are your questions?"}]
2024-09-25 18:12:08 xrx-orchestrator  | [17:12:07 UTC] DEBUG: Sending to agent: {"guid":"35e54838-97b0-4bf2-9f19-7db166fac528","information-received":"{\"name\": \"\", \"date-of-birth\": \"\", \"allergies\": \"\", \"current-medications\": \"\", \"reason-for-visit\": \"\"}"}
2024-09-25 18:12:08 xrx-orchestrator  | [17:12:08 UTC] DEBUG: Raw output: data: {"messages": [{"role": "assistant", "content": "{\n   \"information-received\": {},\n   \"response\": \"To start, could you please tell me your full name?\"\n}"}], "node": "Widget", "output": {"type": "patient-information", "details": "{\"name\": \"\", \"date-of-birth\": \"\", \"allergies\": \"\", \"current-medications\": \"\", \"reason-for-visit\": \"\"}"}, "session": {"guid": "35e54838-97b0-4bf2-9f19-7db166fac528", "information-received": "{\"name\": \"\", \"date-of-birth\": \"\", \"allergies\": \"\", \"current-medications\": \"\", \"reason-for-visit\": \"\"}"}}
2024-09-25 18:12:08 xrx-orchestrator  | 
2024-09-25 18:12:08 xrx-orchestrator  | data: {"messages": [{"role": "assistant", "content": "{\n   \"information-received\": {},\n   \"response\": \"To start, could you please tell me your full name?\"\n}"}], "node": "CustomerResponse", "output": "To start, could you please tell me your full name?", "session": {"guid": "35e54838-97b0-4bf2-9f19-7db166fac528", "information-received": "{\"name\": \"\", \"date-of-birth\": \"\", \"allergies\": \"\", \"current-medications\": \"\", \"reason-for-visit\": \"\"}"}}
2024-09-25 18:12:08 xrx-orchestrator  | 
2024-09-25 18:12:08 xrx-orchestrator  | 
2024-09-25 18:12:08 xrx-orchestrator  | [17:12:08 UTC] DEBUG: Agent response: {"messages":[{"role":"assistant","content":"{\n   \"information-received\": {},\n   \"response\": \"To start, could you please tell me your full name?\"\n}"}],"node":"Widget","output":{"type":"patient-information","details":"{\"name\": \"\", \"date-of-birth\": \"\", \"allergies\": \"\", \"current-medications\": \"\", \"reason-for-visit\": \"\"}"},"session":{"guid":"35e54838-97b0-4bf2-9f19-7db166fac528","information-received":"{\"name\": \"\", \"date-of-birth\": \"\", \"allergies\": \"\", \"current-medications\": \"\", \"reason-for-visit\": \"\"}"}}
2024-09-25 18:12:08 xrx-orchestrator  | [17:12:08 UTC] DEBUG: Handling agent response of type Widget: [object Object], modality: audio
2024-09-25 18:12:08 xrx-orchestrator  | [17:12:08 UTC] DEBUG: Received from Agent:[object Object]
2024-09-25 18:12:08 xrx-orchestrator  | [17:12:08 UTC] DEBUG: Not sending to TTS
2024-09-25 18:12:08 xrx-orchestrator  | [17:12:08 UTC] DEBUG: Agent response: {"messages":[{"role":"assistant","content":"{\n   \"information-received\": {},\n   \"response\": \"To start, could you please tell me your full name?\"\n}"}],"node":"CustomerResponse","output":"To start, could you please tell me your full name?","session":{"guid":"35e54838-97b0-4bf2-9f19-7db166fac528","information-received":"{\"name\": \"\", \"date-of-birth\": \"\", \"allergies\": \"\", \"current-medications\": \"\", \"reason-for-visit\": \"\"}"}}
2024-09-25 18:12:08 xrx-orchestrator  | [17:12:08 UTC] DEBUG: Handling agent response of type CustomerResponse: To start, could you please tell me your full name?, modality: audio
2024-09-25 18:12:08 xrx-orchestrator  | [17:12:08 UTC] DEBUG: Received from Agent:To start, could you please tell me your full name?
2024-09-25 18:12:08 xrx-orchestrator  | [17:12:08 UTC] DEBUG: Sending to TTS:To start, could you please tell me your full name?
2024-09-25 18:12:08 xrx-tts           | INFO:elevenlabs_tts:Connected to 11labs websocket
2024-09-25 18:12:08 xrx-tts           | INFO:elevenlabs_tts:Sent initial message to 11labs
2024-09-25 18:12:08 xrx-tts           | INFO:elevenlabs_tts:Sent EOS message to 11labs
2024-09-25 18:12:08 xrx-tts           | ERROR:elevenlabs_tts:Error from 11labs: voice_id_does_not_exist
2024-09-25 18:12:08 xrx-tts           | ERROR:elevenlabs_tts:An error occurred while receiving data from 11labs
2024-09-25 18:12:08 xrx-tts           | Traceback (most recent call last):
2024-09-25 18:12:08 xrx-tts           |   File "/tts/elevenlabs_tts.py", line 81, in synthesize
2024-09-25 18:12:08 xrx-tts           |     raise Exception(f"ElevenLabs API error: {data['error']}")
2024-09-25 18:12:08 xrx-tts           | Exception: ElevenLabs API error: voice_id_does_not_exist
2024-09-25 18:12:09 xrx-orchestrator  | [17:12:08 UTC] DEBUG: Received from TTS: {"action":"done"}
2024-09-25 18:12:09 xrx-orchestrator  | [17:12:08 UTC] DEBUG: TTS done, sending cached agent responses
alessandro-neri commented 2 days ago

Hi @monkeydust .

You need to add the voice to your "voice library". If you don't, you'll encounter the cryptic "voice_id _does_not_exist" error. To do this, paste the ID from the .env file onto the elevenlabs dashboard and click add.

https://www.reddit.com/r/ElevenLabs/comments/16f22rd/texttospeech_api_for_generated_voices_not_working/