Fix Azure Client /openai/deployments route override to be used more w…

[isebasus]

…idely

• [ ] I understand that this repository is auto-generated and my pull request may not be merged

Changes being requested

Additional context & links

Hello, there seems to be a problem with the openai python client regarding the AzureOpenAI client.

While the Python client operates seamlessly with most endpoints, such as chat/completions (as detailed in the API reference: https://platform.openai.com/docs/api-reference/chat/create), I've identified a discrepancy when it comes to training-related routes.

It appears that the AzureOpenAI Client incorrectly appends /openai to the request URLs for training operations. This modification results in erroneous routes that my Azure deployment does not recognize, as it does not support URLs containing the openai segment.

The routes that get affected are:

• https://platform.openai.com/docs/api-reference/fine-tuning/create
• https://platform.openai.com/docs/api-reference/fine-tuning/list-events
• https://platform.openai.com/docs/api-reference/fine-tuning/list
• https://platform.openai.com/docs/api-reference/files/create
• etc

My fix resolves these issues. Would like someone to take a look.

Thanks

[RobertCraigie]

cc @kristapratico

[kristapratico]

Hi @isebasus - can you share an example of how you're building the Azure client and calling the fine-tuning operations? Fine tuning requests should include the /openai/ segment according to docs so I'm curious what could be happening to create the erroneous routes.

[isebasus]

Here is my client setup:

class OpenAIService:
    def __init__(self, token_manager: TokenManager):
        self.token_manager = token_manager
        self.token_url = "<token_url"
        self.scope = "azureopenai-readwrite"
        self.client = self.__create_client()

    def __create_client(self) -> AzureOpenAI:
        token = self.token_manager.get_oauth_token(self.token_url, CLIENT_ID, CLIENT_SECRET, self.scope)

        openai.api_type = "azure"
        openai.api_base = "<base_url>"
        openai.api_version = "2024-05-01-preview"
        openai.api_key = token

        return AzureOpenAI(
            api_version="2024-05-01-preview",
            api_key=token,
            azure_endpoint="<base_url>",
        )

Could it be something wrong with the AzureOpenAI deployment itself?

[kristapratico]

@isebasus can you share what you're passing for azure_endpoint? Is this just the endpoint to your Azure OpenAI resource, like "https://{resource-name}.openai.azure.com/"?

In terms of not recognizing deployments, another thing to keep in mind is that only a subset of models / model versions support fine-tuning: https://learn.microsoft.com/azure/ai-services/openai/concepts/models?tabs=python-secure%2Cglobal-standard%2Cstandard-chat-completions#fine-tuning-models

When calling fine-tune create, you need to pass the model name (not deployment name) with its model version. So for gpt-4o (2024-08-06), you would pass model=gpt-4o-2024-08-06.

[isebasus]

@kristapratico the Azure deployment is an Internal NVIDIA instance, so I cannot share the azure_endpoint. However yes, it is something like "https://{domain_name}/llm/v1/azure/".

Here is the function I am using to create a training job:

def create_training_job(self, training_path: str) -> str:
        """
        Creates a training job with the OpenAI API using the content of the specified training file.

        :param training_path: The file path to the training data to be uploaded for fine-tuning.
        :return: The ID of the created fine-tuning job.
        """
        try:
            # Upload training file
            file = self.client.files.create( # FAILS AT THIS CALL
                file=open(training_path, "rb"),
                purpose="fine-tune"
            )

            while True:
                file_status = self.client.files.retrieve(file.id) # FAILS AT THIS CALL TOO
                if file_status.status == 'processed':
                    break
                print(f"File: {file.filename} is still uploading...")
                time.sleep(1)

            print(f"File: {file.id} has been uploaded, creating fine_tuning job")

            # Create fine tuned model
            fine_tune_job = self.client.fine_tuning.jobs.create( # FAILS AT THIS CALL TOO
                training_file=file.id,
                model="gpt-4o-2024-08-06"
            )

            return fine_tune_job.id

It looks like we are using gpt-4o-2024-08-06

[kristapratico]

@isebasus gotcha, thanks for the info. The azure_endpoint keyword argument is really meant to work with an Azure OpenAI resource endpoint in the shape of "https://{resource-name}.openai.azure.com/". If you are using a custom endpoint, then you want to pass base_url into AzureOpenAI instead of azure_endpoint. base_url won't append the /openai/ segment. Can you give that a try and let me know if it works for you?

[isebasus]

Hi @kristapratico, sorry for the late response - I just tried out replacing the base_url with the azure_endpoint input parameters and it worked! Thanks for all the help. I think we can close this PR :)

[kristapratico]

Thanks for following up!

@RobertCraigie can you close this? thanks!