Azure OpenAI Support

[a116062]

Many organizations rely on Azure to use OpenAI's models in order to keep the data private. It would be great if this package supported azure. Below is an example of a function I wrote to hit Azure's end-point.

#' Query Azure OpenAI chat completion models
#'
#' @description
#' Queries Azure OpenAI's chat completions models via the Azure RESTful API and returns the result.
#'
#' Set your openai_api_key prior to calling.
#'
#' @param system The initial instruction or cue provided to the model, serving as a starting point or guiding cue for the generative AI model to generate the desired output
#'
#' @param user The prompt or input provided by the user to the generative AI model, guiding the model's output based on the user's specific input or question
#'
#' @param endpoint The specific URL or address where the model is deployed and accessible for making API requests. See Azure portal > "Keys and Endpoint".
#'
#' @param deployment_name The unique identifier or name given to a specific deployment of an OpenAI model on Azure. It helps distinguish and identify different instances or versions of the model that are deployed and accessible through the Azure endpoint.
#'
#' @param api_version The specific version of the API that is used to interact with the OpenAI model hosted on Azure. See "Supported versions" in the Azure OpenAI docs.
#'
#' @param openai_api_key A unique access token that is used to authenticate and authorize API requests to the OpenAI models hosted on Azure.
#'    Find the API key at portal.azure.com -> Azure OpenAI  -> Your Subscription -> Manage Keys -> Copy to Clipboard.
#'    Set the env var by running `Sys.setenv("OPENAI_API_KEY"='Your Copied API Key Here')`.
#'    This key is used by everyone within Progressive to query models deployed on Azure OpenAI.
#'
#' @param temperature A number between 0 and 2 that controls the randomness in the models output.
#'    What sampling temperature to use, between 0 and 2.
#'    Higher values like 0.8 will make the output more random,
#'    while lower values like 0.2 will make it more focused and deterministic.
#'
#'    We recommend altering this or top_p but not both.
#'
#' @param top_p A number between 0 and 1 that controls the diversity of the generated responses.
#'    An alternative to sampling with temperature, called nucleus sampling,
#'    where the model considers the results of the tokens with top_p probability mass.
#'    So 0.1 means only the tokens comprising the top 10% probability mass are considered.
#'
#'    We recommend altering this or temperature but not both.
#'
#' @param presence_penalty
#'    Number between -2.0 and 2.0. Positive values penalize new tokens based on whether they appear in the text so far, increasing the model's likelihood to talk about new topics.
#'
#' @param frequency_penalty
#'     Number between -2.0 and 2.0. Positive values penalize new tokens based on their existing frequency in the text so far, decreasing the model's likelihood to repeat the same line verbatim.
#'
#' @param response_format Either 'text' or 'json'. If 'json', you must also tell the model that you want it to return JSON in the user prompt.
#'
#' @param dry_run If TRUE, returns the http request but don't actually send it.
#'
#' @param return_resp_obj If TRUE, returns the entire response object instead of only the model output.
#'
#' @examples
#' Sys.setenv("OPENAI_API_KEY"='my api key')
#' query_openai_chat("The quick brown fox jumps over the lazy dog. Translate into French.")
#'
#' query_openai_chat(
#'   user = "Who were the first 3 United States Presidents? Return this in JSON format.",
#'   response_format = 'json') %>%
#'   jsonlite::prettify()
#'
#' @source
#' https://learn.microsoft.com/en-us/azure/ai-services/openai/reference#chat-completions
#'
#' @export

query_openai_chat <- function(
    user,
    system = "You are a helpful assistant.",
    endpoint = "https://yourdomain.openai.azure.com/",
    deployment_name = "gpt-4o",
    api_version = "2024-02-01",
    openai_api_key = Sys.getenv("OPENAI_API_KEY"),
    temperature = 1,
    top_p = 1,
    presence_penalty = 0,
    frequency_penalty = 0,
    response_format = 'text',
    dry_run = FALSE,
    return_resp_obj = FALSE) {

  response_format <- match.arg(response_format, choices = c('text','json'))

  # Build the endpoint URL
  full_endpoint <- paste0(endpoint, "openai/deployments/", deployment_name,
                          "/chat/completions?api-version=", api_version)

  messages <- list(
    list(role = "system", content = system),
    list(role = "user", content = user)
  )

  response_format <- case_when(
    response_format == 'json' ~ list('type' = 'json_object'),
    T ~ list('type' = 'text')
  )

  # Convert parameters to JSON format
  messages_json <- jsonlite::toJSON(list(messages = messages,
                                         temperature = temperature,
                                         top_p = top_p,
                                         presence_penalty = presence_penalty,
                                         frequency_penalty = frequency_penalty,
                                         response_format = response_format),
                                    auto_unbox = TRUE)

  # Prepare the request
  req <- httr2::request(full_endpoint) %>%
    httr2::req_headers(
      `Content-Type` = "application/json",
      `api-key` = openai_api_key
    ) %>%
    httr2::req_body_raw(messages_json, "application/json")

  # If asked, return the http request without actually sending
  if (dry_run) return(req %>% httr2::req_dry_run() %>% append(c(body = req$body$data)))

  # Execute the request and parse the response
  resp <- req %>%
    httr2::req_retry(max_tries = 3) %>%
    httr2::req_perform() %>%
    httr2::resp_body_json()

  if (return_resp_obj) return(resp)

  resp$choices[[1]]$message$content
}

[edubruell]

Thanks, for the code example and writing the issue. This looks fairly similar to the vanilla openai API request. My original idea was to add the azure logic into the chatgpt() or my planned openai() function that implements OpenAI Api calls and to just look whether a deployment and an "AZURE-OPENAI-KEY" was set to see if it should direct to azure instead of OpenAI. But there seems to be some more potential to add azure-specific functionality like data_sources in an own function and models seem to be specified differently in the azure API. Perhaps the best choice would be to implement a azure_openai() function instead. I will try to do this, but will need to sign up for azure myself to add this feature. I guess I will need a few weeks to implement this.

[edubruell]

I had a few minutes and implemented a first version of azure_openai() earlier than I had thought and could test it a bit, before the relatively strong rate limiting of the Azure OpenAI free tier cut me off for a day. rate limiting, authentication and error messages are different enough for azure that having azure_openai() as a seperate function from openai() seems reasonable . Here is the documentation on the pkgdown site. Till my ratelmiting kicked in I tested, basic text messages to the API, streaming back tokens and normal json mode (all worked for me). The current environment variable azure_openai() searches for the API key is called "AZURE_OPENAI_API_KEY". Also the .endpoint-arguement of the function searches for an env-variable called "AZURE_ENDPOINT_URL" by default, but can be set freely in the function. If you want you can test whether the function works for you, and whether multimodal allready work or if Azure deals with images differently than the standard openai API. I'll write unit tests sometimes in the next weeks and will also try to read into azure-specific functionality.

[a116062]

It's working both for text and images! Thank you so much for your work. Noting that...

• The azure example has an error. It should say .deployment not .model
• Your logic assumes/requires a backslash in the end point. e.g., 'myendpoint.openai.azure.com/' works but 'myendpoint.openai.azure.com' fails without a helpful error message.

[edubruell]

Thanks a lot for testing it! I now changed the reuqest building to properly use httr2 and both myendpoint.openai.azure.com/ and myendpoint.openai.azure.com should now work. The example is also updated to have .deployment instead of .model. I'll add azure_openai() to the get started vignette then.