dosubot[bot]
commented
1 year ago Answer generated by a π€
Answer
I found related issues that might be helpful. I did my best to summarize the solutions, but I recommend looking at the related issues yourself.
Related Issues
Closed Issues
getting the "This model's maximum context length is 4097 tokens" error using gpt-4 and gpt-3.5-turbo-16k model
The suggested solution is: The user needs to pass the language model (llm) to the toolkit as well, otherwise, it uses the default model.
Issue: Update OpenAI model token mapping to reflect new API update 2023-06-13
This issue was closed by a pull request: Update model token mappings/cost to include 0613 models
Open Issues
summarization
No solution found.
reduce with long text
No discussion on issue
load_qa_chain with chain_type="map_reduce" can not process long document
The suggested solution is:
The issue you're encountering is due to the fact that the input document is too long for the language model's maximum context length. The MapReduceDocumentsChain is designed to process multiple documents, but it doesn't automatically split a single long document into smaller chunks. To solve this issue, you can split the long document into smaller chunks before passing it to the load_qa_chain. You can use the CharacterTextSplitter to split the long document into smaller chunks:
from langchain.document_loaders import TextLoader
from langchain.chains.question_answering import load_qa_chain
from langchain.llms import OpenAI
from langchain.text_splitter import CharacterTextSplitter
loader = TextLoader("state_of_the_union.txt")
document = loader.load()
# Split the long document into smaller chunks
text_splitter = CharacterTextSplitter(chunk_size=1000, chunk_overlap=0)
documents = text_splitter.split_documents([document])
chain = load_qa_chain(OpenAI(temperature=0), chain_type="map_reduce")
query = "What did the president say about Justice Breyer"
chain({"input_documents": documents, "question": query}, return_only_outputs=True)This will split the long document into smaller chunks and then pass them to the load_qa_chain function, allowing it to process the document without exceeding the token limit.
map_reduce._split_list_of_docs has bugs
No solution found.
About map_reduce.py
No solution found.
Update the token_max value in mp_reduce.ValueError:"A single document was so long it could not be combined "γ"A single document was longer than the context length,"
The suggested solution is: The user should create a pull request with the proposed changes to dynamically modify the value of token_max based on the model used. This would enhance the functionality and make it more flexible for different models. The maintainers will then review the contribution and decide if it should be merged into the LangChain framework.
load_qa_chain with map_reduce results in "Token indices sequence length" error
The suggested solution is: Overcome the "Token indices sequence length" error by passing in a custom combine_prompt to load_qa_chain. This error occurs because the default text for the combine prompt in the QA chain is too long for the flan_t5_xxl LLM on HuggingFace. The error messages from LangChain could be improved to make this clearer.
About the token_max variable in the "langchain\chains\combine_documents\map_reduce.py" file.
No solution found.
map_reduce summarizer errors when using LLM with smaller context window
The suggested solution is: Specify the maximum document to construct with the token_max parameter in the run method of the summary chain. For example, summary_chain.run(input_documents=docs, token_max=2000). If you want to automatically use the maximum context size, you can set token_max to -1. If you are using Japanese or any other language, you need to set max_tokens in the OpenAI object.
This response is meant to be useful, save you time, and share context. It is not meant to be a precise solution, but rather a starting point for your own research.
Help me be more useful! Please leave a π if this is helpful and π if it is irrelevant.
System Info
LangChain==0.0.177 Python==3.10
Who can help?
No response
Information
Related Components
Reproduction
When using the map_reduce technique with a GPT 16k model, you receive an error message: 'A single document was longer than the context length, we cannot handle this.' I believe this is due to the token_max limit mentioned in the following context. Is it possible to address this issue? It no longer makes sense to use a 16k model if we cannot send documents exceeding the hardcoded limit of 3000 tokens. Could you please modify this hardcoded value from the 4098 token models to accommodate the 16k models? Thank you.
Expected behavior
I would like be able to give a Document object that would be more than 3000 tokens.