search

Haste171 / langchain-chatbot

AI Chatbot for analyzing/extracting information from data in conversational format.
MIT License
401 stars 88 forks source link

Token Limitation #9

Closed ahgsql closed 1 year ago

ahgsql commented 1 year ago

Got en error about token size. How to limit it to avoid Errors?

openai.error.InvalidRequestError: This model's maximum context length is 4097 tokens. However, your messages resulted in 4599 tokens. Please reduce the length of the messages

Haste171 commented 1 year ago

Reducing the amount of sources used sometimes helps with this problem, not exactly sure why it happens but it's mainly thought to occur because the retrievalchain inputted to much context in a chunk to the llm

Haste171 commented 1 year ago

You can also use gpt-4 with the 8k token limit if you have it

Haste171 commented 1 year ago

Use the map_reduce method