Permission Denied Error when using with_options in OpenAI client

[VRSEN]

Description

When attempting to create a file using the OpenAI Client with custom options set via with_options, a PermissionDeniedError is thrown indicating a missing authentication token in the header.

Steps to Reproduce

1. Initialize the OpenAI Client and patch it.
2. Attempt to create a file using the following code:

   file_id = self.client.with_options(
      timeout=80 * 1000,  # Set a custom timeout
   ).files.create(file=f, purpose="assistants").id

3. Observe the PermissionDeniedError with a 403 status code indicating a missing 'astra-api-token'.

Expected Behavior

The file creation should proceed without any permission errors, utilizing the provided options including the custom timeout.

Actual Behavior

The operation fails with the following error:

PermissionDeniedError: Error code: 403 - {'detail': 'Must pass an astradb token in the astra-api-token header'}

Additional Information

• Removing the with_options method call and directly using files.create() without any options works correctly, suggesting an issue with how headers or tokens are managed when with_options is used.
• OpenAI version: 1.21.2
• Python version: 3.10.13
• Operating System: macOS 14.1.1

[phact]

Hi thanks for sharing your issue!

Please set your api token as an environment variable in your .env file:

# https://astra.datastax.com/ --> tokens --> administrator user --> generate
export ASTRA_DB_APPLICATION_TOKEN="AstraCS..."

or directly in python if you prefer

env.environ['ASTRA_DB_APPLICATION_TOKEN'] = 'AstraCS...'

The patched sdk turns env vars into the required headers to make it easy to swap out third party models in and out.

[phact]

Hi @VRSEN,

Checking in, did this help?

--Seb

[VRSEN]

Hey @phact, no last time I tested it, it didn't help. However, this did:

from openai import OpenAI
from streaming_assistants import patch
client = patch(OpenAI(default_headers={
    'astra-api-token': os.environ['ASTRA_DB_APPLICATION_TOKEN'],
  }
))

[phact]

Interesting, we pull it in the patch here:

https://github.com/phact/streaming-assistants/blob/main/streaming_assistants/patch.py#L492

but we only do it for the calls that should technically need it (calls that require database persistence like messages, threads, assistants, etc.). Completions for example, don't need persistence so we don't look for the env var.

It's possible that there's a path we missed. Do you remember exactly what you were running to get the error?

[VRSEN]

I was running this on my own framework Agency-Swarm v0.1.7, which uses the with_options method to create files as mentioned in the issue. Here’s an example of setting up a basic agent:

from openai import OpenAI
from streaming_assistants import patch
from agency_swarm import Agent, Agency, set_openai_client

client = patch(OpenAI(default_headers={
    'astra-api-token': os.environ['ASTRA_DB_APPLICATION_TOKEN'],
    "OpenAI-Beta": "assistants=v1"
}))

set_openai_client(client)

ceo = Agent(name="CEO",
            description="Responsible for client communication, task planning, and management.",
            instructions="Please communicate with users and other agents.",
            model="anthropic/claude-3-haiku-20240307",
            files_folder="./files",
            tools=[])

agency = Agency([ceo])

Ensure you have files in your files folder. Removing 'astra-api-token': os.environ['ASTRA_DB_APPLICATION_TOKEN'] from the headers will cause the setup to fail.

[phact]

"OpenAI-Beta": "assistants=v1"

There is no way of overriding the OpenAI-Beta header once you've gone above sdk v1.21.0 The sdk will stomp your default header (which makes sense because it's using all the new types).

I'll test this again once we ship v2 support.

[phact]

So I shipped V2 this weekend and briefly looked this. It currently blows up like so:

$ poetry run python agency-swarm-test/main.py 
...
openai.UnprocessableEntityError: Error code: 422 - {'detail': [{'type': 'model_attributes_type', 'loc': ['body', 'response_format'], 'msg': 'Input should be a valid dictionary or object to extract fields from', 'input': 'auto'}]}

~response_format is an instructor thing. We'll have to look into how agency-swarm uses the instructor library and assistants at the same time.~

Fixed this particular issue and released in v0.2.1.

I can now repro the original issue, it's a problem with the client library and the fix will get released in the next few minutes.

cc: @VRSEN

[phact]

fixed in the client