uezo / aiavatarkit

🥰 Building AI-based conversational avatars lightning fast ⚡️💬
Apache License 2.0
162 stars 10 forks source link

Add experimental support for OpenAI audio APIs #48

Closed uezo closed 1 month ago

uezo commented 1 month ago

Add multilingual support for SpeechListeners using OpenAI's Speech-to-Text and SpeechController using Text-to-Speech. This update allows to handle both speech recognition and voice output in multiple languages dynamically, without needing explicit and static language settings.

Usage:

from aiavatar import AIAvatar
from aiavatar.device import AudioDevice
from aiavatar.listeners.openailisteners import (
    OpenAIWakewordListener,
    OpenAIVoiceRequestListener
)
from aiavatar.speech.openaispeech import OpenAISpeechController

# Get default audio devices
devices = AudioDevice()

# Speech
speech_controller = OpenAISpeechController(
    api_key=OPENAI_API_KEY,
    device_index=devices.output_device
)

# Wakeword
async def on_wakeword(text):
    await app.start_chat(request_on_start=text, skip_start_voice=True)

wakeword_listener = OpenAIWakewordListener(
    api_key=OPENAI_API_KEY,
    device_index=devices.input_device,
    wakewords=["こんにちは"],
    on_wakeword=on_wakeword
)

# Request
request_listener = OpenAIVoiceRequestListener(
    api_key=OPENAI_API_KEY,
    device_index=devices.input_device
)

# Create AIAvatar with OpenAI Components
app = AIAvatar(
    openai_api_key=OPENAI_API_KEY,
    wakeword_listener=wakeword_listener,
    request_listener=request_listener,
    speech_controller=speech_controller,
    noise_margin=10.0,
    verbose=True
)
app.start_listening_wakeword()