Issues with Azure OpenAI

[ushakrishnan]

System Info

OS = MACOS langchain=0.0.179 (also tried 0.0.174 and 0.0.178)

Who can help?

@hwchase17 @agola11

The full code below is single file. imports and other information not added to keep it crisp.

The following works with no issues:

llm = AzureOpenAI(openai_api_base=openai_api_base , model="text-davinci-003",  engine="text-davinci-003", temperature=0.1, verbose=True, deployment_name="text-davinci-003", deployment_id="text-davinci-003", openai_api_key=openai_api_key)

resp = llm("Tell me pub joke")
print(resp)

The following does not work.

#get document store
store = getfromstore(collection_name="sou_coll")

# Create vectorstore info object - metadata repo?
vectorstore_info = VectorStoreInfo(
    name="sou",
    description="sou folder",
    vectorstore=store
)
# Convert the document store into a langchain toolkit
toolkit = VectorStoreToolkit(vectorstore_info=vectorstore_info)

# Add the toolkit to an end-to-end LC
agent_executor = create_vectorstore_agent(
    llm=llm,
    toolkit=toolkit,
    verbose=True
)
response = agent_executor.run(prompt)
print(response)

I can confirm the document store exists and the same code with appropriate OpenAI (not Azure OpenAI) works as expected with no issue. Azure OpenAI gives the following error -

File "/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages/openai/api_resources/abstract/engine_api_resource.py", line 83, in __prepare_create_request
    raise error.InvalidRequestError(
openai.error.InvalidRequestError: Must provide an 'engine' or 'deployment_id' parameter to create a <class 'openai.api_resources.completion.Completion'>

Observation LLM is correct since the first part (write a joke) works. The agent does not. Please help!

Information

• [ ] The official example notebooks/scripts
• [ ] My own modified scripts

Related Components

• [X] LLMs/Chat Models
• [ ] Embedding Models
• [ ] Prompts / Prompt Templates / Prompt Selectors
• [ ] Output Parsers
• [ ] Document Loaders
• [X] Vector Stores / Retrievers
• [ ] Memory
• [X] Agents / Agent Executors
• [X] Tools / Toolkits
• [ ] Chains
• [ ] Callbacks/Tracing
• [ ] Async

Reproduction

!/usr/bin/env python3

import sys from dotenv import load_dotenv

Load default environment variables (.env)

load_dotenv()

Import os to set API key

import os

Import OpenAI as main LLM service

from langchain.llms import AzureOpenAI from langchain.callbacks import get_openai_callback

Bring in streamlit for UI/app interface

import streamlit as st

Import PDF document loaders...there's other ones as well!

from langchain.document_loaders import PyPDFLoader

Import chroma as the vector store

from langchain.vectorstores import Chroma

from common.funs import getfromstore

Import vector store stuff

from langchain.agents.agent_toolkits import ( create_vectorstore_agent, VectorStoreToolkit, VectorStoreInfo )

Set this to azure

openai_api_type = os.environ["OPENAI_API_TYPE"] ="azure" openai_api_version = os.environ["OPENAI_API_VERSION"] = os.environ["AOAI_OPENAI_API_VERSION"] openai_api_base = os.environ["OPENAI_API_BASE"] = os.environ["AOAI_OPENAI_API_BASE"] openai_api_key = os.environ["OPENAI_API_KEY"] = os.environ["AOAI_OPENAI_API_KEY"]

Create instance of OpenAI LLM

llm = AzureOpenAI(openai_api_base=openai_api_base , model="text-davinci-003", temperature=0.1, verbose=True, deployment_name="text-davinci-003", openai_api_key=openai_api_key)

llm = AzureOpenAI(openai_api_base=openai_api_base , model="text-davinci-003", engine="text-davinci-003", temperature=0.1, verbose=True, deployment_name="text-davinci-003", deployment_id="text-davinci-003", openai_api_key=openai_api_key)

resp = llm("Tell me pub joke") print(resp) print("------------") st.write(resp) st.write("----------------------")

get document store

store = getfromstore(collection_name="sou_coll")

print(store1.get(["metadatas"]))

Create vectorstore info object - metadata repo?

vectorstore_info = VectorStoreInfo( name="sou", description="sou folder", vectorstore=store )

Convert the document store into a langchain toolkit

toolkit = VectorStoreToolkit(vectorstore_info=vectorstore_info)

Add the toolkit to an end-to-end LC

agent_executor = create_vectorstore_agent( llm=llm, toolkit=toolkit, verbose=True )

st.title("🦜🔗🤗 What would you like to know?") st.write("This sample uses Azure OpenAI")

Create a text input box for the user

prompt = st.text_input('Input your prompt here:')

If the user hits enter

if prompt: with get_openai_callback() as cb:

try:

        # Then pass the prompt to the LLM
        response = agent_executor.run(prompt)
        # ...and write it out to the screen
        st.write(response)
        st.write(cb)
    #except Exception as e:
    #    st.warning
    #    st.write("That was a difficult question!  I choked on it!!  Can you please try again with rephrasing it a bit?")
    #    st.write(cb)
    #    print(e)

# Find the relevant pages
search = store.similarity_search_with_score(prompt)
# Write out the first 
try:
    st.write("This information was found in:")
    for doc in search:
        score = doc[1]
        try:
            page_num = doc[0].metadata['page']
        except:
            page_num = "txt snippets"
        source = doc[0].metadata['source']
        # With a streamlit expander  
        with st.expander("Source: " + str(source) + " - Page: " + str(page_num) + "; Similarity Score: " + str(score) ):
            st.write(doc[0].page_content)
except:
    print("unable to get source document detail")

Expected behavior

The video shows the expected output - https://www.youtube.com/watch?v=q27RbxcfGvE

The OpenAI code in this sample is exact except for changes to LLM and env variables - file https://github.com/ushakrishnan/SearchWithOpenAI/blob/main/pages/6_Q%26A_with_Open_AI.py.

[felipeff]

@ushakrishnan - The VectorStoreToolkit() class accepts an LLM parameter for you to specify your LLM. If you don't give a value by default it creates a new LLM object using the OpenAI class instead of AzureOpenAI with all the default values. You can see this here: https://python.langchain.com/en/latest/_modules/langchain/agents/agent_toolkits/vectorstore/toolkit.html#VectorStoreToolkit

To fix your issue just update your code from:

toolkit = VectorStoreToolkit(vectorstore_info=vectorstore_info)

to:

toolkit = VectorStoreToolkit(vectorstore_info=vectorstore_info, llm=llm)

[ushakrishnan]

Perfect, thank you very much. Worked like a charm!