Exception: OpenAI S2S connection closed unexpectedly

[arkodeep3404]

i get this error

(today) arkodeepchatterjee@Arkodeeps-MacBook-Air today % python3 agent.py connect --room my-room
2024-10-21 20:34:49,346 - DEBUG asyncio - Using selector: KqueueSelector 
2024-10-21 20:34:49,347 - DEV  livekit.agents - Watching /Users/arkodeepchatterjee/Desktop/today 
2024-10-21 20:34:49,725 - DEBUG asyncio - Using selector: KqueueSelector 
2024-10-21 20:34:49,728 - INFO livekit.agents - starting worker {"version": "0.10.2", "rtc-version": "0.17.5"}
2024-10-21 20:34:50,680 - INFO livekit.agents - registered worker {"id": "AW_7nPLUg6MNDot", "region": "", "protocol": 14, "node_id": "ND_LyC5ZDiqyX6Z"}
2024-10-21 20:34:50,681 - INFO livekit.agents - connecting to room my-room 
2024-10-21 20:34:51,603 - INFO livekit.agents - received job request {"job_id": "AJ_36h3t5brLLAS", "dispatch_id": "AD_pGyVXv8LPGvU", "room_name": "my-room", "agent_name": "", "resuming": false}
2024-10-21 20:34:51,635 - INFO livekit.agents - received job request {"job_id": "AJ_bjaNW4rj3kf9", "dispatch_id": "", "room_name": "my-room", "agent_name": "", "resuming": false}
2024-10-21 20:34:52,242 - DEBUG asyncio - Using selector: KqueueSelector {"pid": 52420}
2024-10-21 20:34:52,243 - INFO livekit.agents - initializing process {"pid": 52420}
2024-10-21 20:34:52,243 - INFO livekit.agents - process initialized {"pid": 52420}
2024-10-21 20:34:52,244 - INFO my-worker - starting entrypoint {"pid": 52420, "job_id": "AJ_36h3t5brLLAS"}
2024-10-21 20:34:52,626 - DEBUG asyncio - Using selector: KqueueSelector {"pid": 52421}
2024-10-21 20:34:52,628 - INFO livekit.agents - initializing process {"pid": 52421}
2024-10-21 20:34:52,628 - INFO livekit.agents - process initialized {"pid": 52421}
2024-10-21 20:34:52,629 - INFO my-worker - starting entrypoint {"pid": 52421, "job_id": "AJ_bjaNW4rj3kf9"}
2024-10-21 20:34:55,837 - DEBUG livekit.agents - http_session(): creating a new httpclient ctx {"pid": 52420, "job_id": "AJ_36h3t5brLLAS"}
2024-10-21 20:34:56,132 - DEBUG livekit.agents - http_session(): creating a new httpclient ctx {"pid": 52421, "job_id": "AJ_bjaNW4rj3kf9"}
2024-10-21 20:34:57,055 - ERROR livekit.plugins.openai.realtime - OpenAI S2S error {'type': 'error', 'event_id': 'event_AKoFkgQGluOnCWGHP8Eo2', 'error': {'type': 'invalid_request_error', 'code': 'access_not_enabled', 'message': "Your organization does not have access to the Realtime API. We're in the process of rolling out to all developers.", 'param': None, 'event_id': None}} {"session_id": "not-connected", "pid": 52420, "job_id": "AJ_36h3t5brLLAS"}
2024-10-21 20:34:57,056 - ERROR livekit.plugins.openai.realtime - Error in _recv_task
Traceback (most recent call last):
  File "/Users/arkodeepchatterjee/Desktop/today/today/lib/python3.12/site-packages/livekit/agents/utils/log.py", line 16, in async_fn_logs
    return await fn(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/arkodeepchatterjee/Desktop/today/today/lib/python3.12/site-packages/livekit/plugins/openai/realtime/realtime_model.py", line 833, in _recv_task
    raise Exception("OpenAI S2S connection closed unexpectedly")
Exception: OpenAI S2S connection closed unexpectedly {"pid": 52420, "job_id": "AJ_36h3t5brLLAS"}
2024-10-21 20:34:57,059 - ERROR livekit.plugins.openai.realtime - Error in _main_task
Traceback (most recent call last):
  File "/Users/arkodeepchatterjee/Desktop/today/today/lib/python3.12/site-packages/livekit/agents/utils/log.py", line 16, in async_fn_logs
    return await fn(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/arkodeepchatterjee/Desktop/today/today/lib/python3.12/site-packages/livekit/plugins/openai/realtime/realtime_model.py", line 900, in _main_task
    await asyncio.gather(*tasks)
  File "/Users/arkodeepchatterjee/Desktop/today/today/lib/python3.12/site-packages/livekit/agents/utils/log.py", line 16, in async_fn_logs
    return await fn(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/arkodeepchatterjee/Desktop/today/today/lib/python3.12/site-packages/livekit/plugins/openai/realtime/realtime_model.py", line 833, in _recv_task
    raise Exception("OpenAI S2S connection closed unexpectedly")
Exception: OpenAI S2S connection closed unexpectedly {"pid": 52420, "job_id": "AJ_36h3t5brLLAS"}
2024-10-21 20:34:57,305 - ERROR livekit.plugins.openai.realtime - OpenAI S2S error {'type': 'error', 'event_id': 'event_AKoFlqov0WMlMEmvkP0QC', 'error': {'type': 'invalid_request_error', 'code': 'access_not_enabled', 'message': "Your organization does not have access to the Realtime API. We're in the process of rolling out to all developers.", 'param': None, 'event_id': None}} {"session_id": "not-connected", "pid": 52421, "job_id": "AJ_bjaNW4rj3kf9"}
2024-10-21 20:34:57,305 - ERROR livekit.plugins.openai.realtime - Error in _recv_task
Traceback (most recent call last):
  File "/Users/arkodeepchatterjee/Desktop/today/today/lib/python3.12/site-packages/livekit/agents/utils/log.py", line 16, in async_fn_logs
    return await fn(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/arkodeepchatterjee/Desktop/today/today/lib/python3.12/site-packages/livekit/plugins/openai/realtime/realtime_model.py", line 833, in _recv_task
    raise Exception("OpenAI S2S connection closed unexpectedly")
Exception: OpenAI S2S connection closed unexpectedly {"pid": 52421, "job_id": "AJ_bjaNW4rj3kf9"}
2024-10-21 20:34:57,307 - ERROR livekit.plugins.openai.realtime - Error in _main_task
Traceback (most recent call last):
  File "/Users/arkodeepchatterjee/Desktop/today/today/lib/python3.12/site-packages/livekit/agents/utils/log.py", line 16, in async_fn_logs
    return await fn(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/arkodeepchatterjee/Desktop/today/today/lib/python3.12/site-packages/livekit/plugins/openai/realtime/realtime_model.py", line 900, in _main_task
    await asyncio.gather(*tasks)
  File "/Users/arkodeepchatterjee/Desktop/today/today/lib/python3.12/site-packages/livekit/agents/utils/log.py", line 16, in async_fn_logs
    return await fn(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/arkodeepchatterjee/Desktop/today/today/lib/python3.12/site-packages/livekit/plugins/openai/realtime/realtime_model.py", line 833, in _recv_task
    raise Exception("OpenAI S2S connection closed unexpectedly")
Exception: OpenAI S2S connection closed unexpectedly {"pid": 52421, "job_id": "AJ_bjaNW4rj3kf9"}

with this code

from __future__ import annotations
import logging
from typing import Annotated
from dotenv import load_dotenv
import os

import smtplib
from email.message import EmailMessage

from livekit.agents import (
    AutoSubscribe,
    JobContext,
    WorkerOptions,
    WorkerType,
    cli,
    llm,
    multimodal,
)
from livekit.plugins import openai

load_dotenv()
logger = logging.getLogger("my-worker")
logger.setLevel(logging.INFO)

# print(os.getenv("LIVEKIT_URL"))
# print(os.getenv("LIVEKIT_API_KEY"))
# print(os.getenv("LIVEKIT_API_SECRET"))

def send_email(subject, body, to_email):
    msg = EmailMessage()
    msg.set_content(body)
    msg["Subject"] = subject
    msg["From"] = os.getenv("EMAIL")
    msg["To"] = to_email

    with smtplib.SMTP_SSL(os.getenv("SMTP"), 465) as smtp:
        smtp.login(os.getenv("EMAIL"), os.getenv("PASSWORD"))
        smtp.send_message(msg)

async def entrypoint(ctx: JobContext):
    logger.info("starting entrypoint")

    fnc_ctx = llm.FunctionContext()

    @fnc_ctx.ai_callable()
    async def emergency_email(
        location: Annotated[
            str,
            llm.TypeInfo(description="The location where to send help in emergency"),
        ],
    ):
        """Called when user asks to send help email in emergency. This function will send help email in emergency."""
        logger.info("sending help email in emergency")
        send_email(
            f"Urgent Help Needed at {location}",
            f"This is an emergency situation at {location}. Immediate assistance is required. Please send help as soon as possible to address the critical situation.",
            "arkodeep3404@gmail.com",
        )
        return "emergency help email sent successfully"

    await ctx.connect(auto_subscribe=AutoSubscribe.AUDIO_ONLY)
    participant = await ctx.wait_for_participant()

    # agent = multimodal.MultimodalAgent(
    #     model=openai.realtime.RealtimeModel.with_azure(
    #         azure_deployment=os.getenv("AZURE_OPENAI_MODEL_DEPLOYMENT"),
    #         azure_endpoint=os.getenv("AZURE_OPENAI_ENDPOINT"),
    #         api_key=os.getenv("AZURE_OPENAI_API_KEY"),
    #         api_version=os.getenv("AZURE_OPENAI_API_VERSION"),
    #         voice="alloy",
    #         temperature=0.8,
    #         instructions="You are a helpful assistant who has access to emergency_email function to send emails in emergency",
    #         turn_detection=openai.realtime.ServerVadOptions(
    #             threshold=0.6, prefix_padding_ms=200, silence_duration_ms=500
    #         ),
    #     ),
    #     fnc_ctx=fnc_ctx,
    # )

    agent = multimodal.MultimodalAgent(
        model=openai.realtime.RealtimeModel(
            voice="alloy",
            temperature=0.8,
            instructions="You are a helpful assistant",
            turn_detection=openai.realtime.ServerVadOptions(
                threshold=0.6, prefix_padding_ms=200, silence_duration_ms=500
            ),
        ),
        fnc_ctx=fnc_ctx,
    )

    agent.start(ctx.room, participant)

if __name__ == "__main__":
    cli.run_app(WorkerOptions(entrypoint_fnc=entrypoint, worker_type=WorkerType.ROOM))

anybody knows why????

[GergesBernaba1]

i think this is the same error:

ERROR livekit.plugins.openai.realtime - OpenAI S2S error {'type': 'error', 'event_id': 'event_ANGkyIaX34YNPy9NQphGm', 'error': {'type': 'invalid_request_error', 'code': 'access_not_enabled', 'message': "Your organization does not have access to the Realtime API. We're in the process of rolling out to all developers.", 'param': None, 'event_id': None}} {"session_id": "not-connected"}2024-10-28 12:55:20,488 - ERROR livekit.plugins.openai.realtime - Error in _recv_task Traceback (most recent call last):

if anyone can help please!