pipecat-ai / pipecat

Open Source framework for voice and multimodal conversational AI
BSD 2-Clause "Simplified" License
2.91k stars 219 forks source link

UnknownCallClientError + 11Labs Error #63

Closed hackgoofer closed 5 months ago

hackgoofer commented 5 months ago

Trying to run the chatbot.py, and encountered the following errors. I think there are two errors:

  1. daily core error on unknown call client
  2. 11labs error. I have a free account with 11labs but I have not used any of the free credits. so it should work. I can probably debug the 11labs error separately, and it is likely not super relevant to daily.

I have populated the env variables. And here is the shape of my env:

OPENAI_API_KEY=sk-xxxx                                                                                                 
ELEVENLABS_API_KEY=xxx
ELEVENLABS_VOICE_ID=CnV6BQOHeZCIv4McSXDH
DAILY_SAMPLE_ROOM_URL=https://xx.daily.co/xx-x
DAILY_API_KEY=xxxx

Detailed error

(smol) sasha@iSashair daily-ai-sdk % python src/examples/starter-apps/chatbot.py             
Using cache found in /Users/sasha/.cache/torch/hub/snakers4_silero-vad_master
Eleven Labs API Key: xxx
Eleven Labs Voice ID: xxx
20 2024-03-16 14:30:07,548 🎬 Starting frame consumer thread
{"timestamp":"2024-03-16T21:30:07.577665Z","level":"ERROR","fields":{"message":"startTranscription (request 3) encountered an error: Transcription(Properties(UnknownCallClientError))"},"target":"daily_core::native::ffi::call_client"}
40 2024-03-16 14:30:07,578 on_error: Error handling startTranscription: Transcription(Properties(UnknownCallClientError))
!!! in here, pipeline.source is <Queue maxsize=0 _getters[1]>
10 2024-03-16 14:30:17,075 Generating chat via openai: [{"content": "You are Chatbot, a friendly, helpful robot. Your goal is to demonstrate your capabilities in a succinct way. Your output will be converted to audio. Respond to what the user said in a creative and helpful way, but keep your responses brief. Start by introducing yourself.", "role": "system", "name": "system"}]
[W NNPACK.cpp:64] Could not initialize NNPACK! Reason: Unsupported hardware.
20 2024-03-16 14:30:18,795 === OpenAI LLM TTFB: 1.7094330787658691
40 2024-03-16 14:30:19,559 audio fetch status code: 401, error: <bound method ClientResponse.text of <ClientResponse(https://api.elevenlabs.io/v1/text-to-speech/CnV6BQOHeZCIv4McSXDH/stream?output_format=pcm_16000&optimize_streaming_latency=2) [401 Unauthorized]>
<CIMultiDictProxy('Date': 'Sat, 16 Mar 2024 21:30:18 GMT', 'Server': 'uvicorn', 'Content-Length': '476', 'Content-Type': 'application/json', 'Access-Control-Allow-Origin': '*', 'Access-Control-Allow-Headers': '*', 'Access-Control-Allow-Methods': 'POST, OPTIONS, DELETE, GET, PUT', 'Access-Control-Max-Age': '600', 'strict-transport-security': 'max-age=31536000; includeSubDomains', 'Via': '1.1 google', 'Alt-Svc': 'h3=":443"; ma=2592000,h3-29=":443"; ma=2592000')>
>
hackgoofer commented 5 months ago

It turns out "Money solves all issues". I switched to a paid 11Labs API key and the 11Labs error went away when I ran: python src/examples/foundational/02-llm-say-one-thing.py, but the daily error still persists when running the chatbot.py

kwindla commented 5 months ago

And I think it's a "money solves all issues, or at least a credit card does" on the Daily side of things, too.

We gate transcription behind adding a credit card to a "free" Daily account. We should think through that for users of the dailyai framework. It's definitely extra friction.

Once you add a credit card you get $15 of Daily credit and I think a certain number of free transcription minutes per month. I'll look that up and post the details here.

We'll also think about how to remove this friction from the getting started experience for dailyai.

hackgoofer commented 5 months ago

Confirmed, "money solves all issues"