Reproducible output feature with Azure Open AI (support for seed parameter)

🤖

Hello @sowsan! I'm Dosu, an AI assistant here to help you with your bugs, questions, and contributions to the LangChain repository while we wait for a human maintainer. Don't hesitate to ask if you need any assistance.

Thank you for your feature request. I understand the importance of deterministic output for generative AI applications.

Currently, LangChain does support integration with Azure Open AI as evident from the AzureOpenAI class in the codebase. However, the deterministic output feature you're referring to is controlled by the temperature parameter in the OpenAI API call.

In the current implementation of LangChain, the temperature parameter is a property of the AzureOpenAI class and can be set when initializing an object of this class. Here is the relevant code snippet:

@property
def _default_params(self) -> Dict[str, Any]:
    """Get the default parameters for calling OpenAI API."""
    normal_params: Dict[str, Any] = {
        "temperature": self.temperature,
        "top_p": self.top_p,
        "frequency_penalty": self.frequency_penalty,
        "presence_penalty": self.presence_penalty,
        "n": self.n,
        "logit_bias": self.logit_bias,
    }

    if self.max_tokens is not None:
        normal_params["max_tokens"] = self.max_tokens
    if self.request_timeout is not None and not is_openai_v1():
        normal_params["request_timeout"] = self.request_timeout

    # Azure gpt-35-turbo doesn't support best_of
    # don't specify best_of if it is 1
    if self.best_of > 1:
        normal_params["best_of"] = self.best_of

    return {**normal_params, **self.model_kwargs}

To make the output deterministic, you would set self.temperature = 0 when initializing the AzureOpenAI object.

However, if there are additional parameters required by Azure Open AI to enable deterministic output that are not currently supported by LangChain, we would need to extend the AzureOpenAI class to include these parameters.

If you're willing to contribute to this change, you could start by extending the AzureOpenAI class to include any additional parameters required for deterministic output. Once you've made these changes, you could test them by initializing an AzureOpenAI object with the new parameters and checking if the output is deterministic.

Please let me know if you need further clarification or assistance. I'm here to help!

Best regards, Dosu

Sources

libs/langchain/langchain/llms/openai.py

This response is meant to be useful and save you time. It is not meant to be a precise solution, but rather a starting point for your own research.

Help me be more useful! Please leave a 👍 if this is helpful and 👎 if it is irrelevant.

If you want to continue the conversation, start your reply with @dosu-bot.

langchain-ai / langchain

Reproducible output feature with Azure Open AI (support for seed parameter) #14850

Feature request

Motivation

Your contribution

Sources