RateLimitError

[yazanrisheh]

@dosu-bot When I use chromadb instead of deeplake, my code worked fine but with deeplake, I am facing this error when I use deeplake for some reason despite everything is the exact same thing: creating embeddings: 12%|██████████▏ | 3/26 [00:02<00:17, 1.32it/s]Retrying langchain.embeddings.openai.embed_with_retry.._embed_with_retry in 4.0 seconds as it raised RateLimitError: Rate limit reached for text-embedding-ada-002 in organization org-m0YReKtLXxUATOVCwzcBNfqm on requests per min. Limit: 3 / min. Please try again in 20s. Contact us through our help center at help.openai.com if you continue to have issues. Please add a payment method to your account to increase your rate limit. Visit https://platform.openai.com/account/billing to add a payment method.. Retrying langchain.embeddings.openai.embed_with_retry.._embed_with_retry in 4.0 seconds as it raised RateLimitError: Rate limit reached for text-embedding-ada-002 in organization org-m0YReKtLXxUATOVCwzcBNfqm on requests per min. Limit: 3 / min. Please try again in 20s. Contact us through our help center at help.openai.com if you continue to have issues. Please add a payment method to your account to increase your rate limit. Visit https://platform.openai.com/account/billing to add a payment method..

Below is my code:

from langchain.chat_models import ChatOpenAI from langchain.prompts.prompt import PromptTemplate from langchain.memory import ConversationBufferMemory from langchain.chains import ConversationalRetrievalChain from langchain.document_loaders import WebBaseLoader from langchain.text_splitter import RecursiveCharacterTextSplitter from langchain.vectorstores import Qdrant from langchain.callbacks import StreamingStdOutCallbackHandler from langchain.embeddings import OpenAIEmbeddings from langchain.callbacks import get_openai_callback from langchain.vectorstores import Chroma from langchain.vectorstores import DeepLake from langchain.document_loaders import DirectoryLoader, PyPDFLoader from dotenv import load_dotenv import time import warnings

warnings.filterwarnings("ignore")

load_dotenv() directory_path = "C:\Users\Asus\Documents\Vendolista" pdf_loader = DirectoryLoader(directory_path, glob="*/.pdf", show_progress=True, use_multithreading=True, silent_errors=True, loader_cls = PyPDFLoader)

documents = pdf_loader.load() print(str(len(documents))+ " documents loaded")

def print_letter_by_letter(text): for char in text: print(char, end='', flush=True) time.sleep(0.02)

Create embeddings

def langchain(customer_prompt, chat_history):

llm = ChatOpenAI(temperature = 0, model_name='gpt-3.5-turbo', callbacks=[StreamingStdOutCallbackHandler()], streaming = True)

text_splitter = RecursiveCharacterTextSplitter(
    chunk_size=800,
    chunk_overlap=80,
)
chunks = text_splitter.split_documents(documents)

embeddings = OpenAIEmbeddings()
persist_directory = "C:\\Users\\Asus\\OneDrive\\Documents\\Vendolista"
knowledge_base = Chroma.from_documents(chunks, embeddings, persist_directory = persist_directory)
#save to disk
knowledge_base.persist()
#To delete the DB we created at first so that we can be sure that we will load from disk as fresh db
knowledge_base = None
new_knowledge_base = Chroma(persist_directory = persist_directory, embedding_function = embeddings)

# Create a custom prompt for your use case
prompt_template = """
Answer the Question as a AI assistant that is answering based on the documents only. If the question is unrelated
then say "sorry this question is completely not related. If you think it is, email the staff
and they will get back to you: yazanrisheh@hotmail.com." Do not ever answer with "I don't know" to any question.
You either give an answer or mention it's not related.

Text: {context}

Question: {question}
Answer :
"""
PROMPT = PromptTemplate(
    template=prompt_template, input_variables=["context", "question"]
)

memory = ConversationBufferMemory(llm=llm, memory_key='chat_history', input_key='question', output_key='answer', return_messages=False)
conversation = ConversationalRetrievalChain.from_llm(
    llm=llm,
    retriever=new_knowledge_base.as_retriever(),
    memory=memory,
    chain_type="stuff",
    verbose=True,
    combine_docs_chain_kwargs={"prompt":PROMPT}
)

return conversation({"question": customer_prompt, "chat_history": memory})

def main(): chat_history = []

while True:
    customer_prompt = input("Ask me anything about the files (type 'exit' to quit): ")

    if customer_prompt.lower() in ["exit"] and len(customer_prompt) == 4:
        end_chat = "Thank you for visiting us! Have a nice day"
        print_letter_by_letter(end_chat)
        break

    if customer_prompt:
        with get_openai_callback() as cb:
            response = (langchain(customer_prompt, chat_history))
        print(response['answer'])
        print(cb)

if name == "main": main()

[dosubot[bot]]

🤖

Hello,

The error message you're seeing is due to hitting the rate limit for the OpenAI API. This limit is set by OpenAI and restricts the number of requests you can make to the API within a certain time period. When this limit is exceeded, the API responds with a RateLimitError, which is what you're seeing.

There are a few ways you can handle this:

1. Add a payment method to your OpenAI account: As suggested in the error message, adding a payment method to your OpenAI account can increase your rate limit. You can do this by visiting the billing section of your account on the OpenAI platform.

2. Implement retry logic: LangChain has built-in support for retrying requests when a RateLimitError is encountered. This is handled by the _create_retry_decorator function, which creates a retry decorator that will retry the request when certain errors, including RateLimitError, are encountered. The maximum number of retries is specified by the max_retries attribute of the BaseOpenAI or OpenAIChat object. You can find more details about this in the LangChain repository.

3. Adjust the batch size: Another way to avoid rate limit errors is to adjust the batch size in the Language Learning Model (LLM) used. This can be done by setting the batch size on the LLM. For example:

llm = OpenAI(batch_size=5, temperature=0)

This code sets the batch size to 5 when using the OpenAI model. Adjusting this parameter to a suitable value should help avoid rate limit errors.

As for the difference between Chroma and DeepLake, I wasn't able to find specific information within the LangChain repository. However, both are vector stores used in LangChain for storing and retrieving document embeddings. The choice between them may depend on your specific use case and requirements.

I hope this helps! Let me know if you have any other questions.

Sources

• RateLimitError
• Langchain rate limit error while invoking OpenAI API
• Langchain QA over large documents results in Rate limit errors
• libs/langchain/langchain/llms/openai.py

---

This response is meant to be useful and save you time. It is not meant to be a precise solution, but rather a starting point for your own research.

Help me be more useful! Please leave a 👍 if this is helpful and 👎 if it is irrelevant.

If you want to continue the conversation, start your reply with @dosu-bot.

[dosubot[bot]]

Hi, @yazanrisheh

I'm helping the LangChain team manage their backlog and am marking this issue as stale. From what I understand, you raised an issue regarding a RateLimitError encountered when using deeplake for text embedding. I provided a detailed response explaining the cause of the error and offering solutions, such as adding a payment method to the OpenAI account, implementing retry logic, and adjusting the batch size. It seems that the issue has been resolved with the provided solutions.

Could you please confirm if this issue is still relevant to the latest version of the LangChain repository? If it is, please let the LangChain team know by commenting on the issue. Otherwise, feel free to close the issue yourself, or it will be automatically closed in 7 days.

Thank you for your understanding and cooperation.