Open xfocus3 opened 1 year ago
faddy19
commented
1 year ago I have the same issue.
InvalidRequestError: This model's maximum context length is 4097 tokens. However, your messages resulted in 8776 tokens. Please reduce the length of the messages.
Anyone a solution or adjustment in the code to prevent that?
Connoropolous
commented
1 year ago Also have this issue and haven’t discovered a fox yet.
Connoropolous
commented
1 year ago I didn’t experience the issue when using the initial commit on the repo, so it seems like a flaw that’s been introduced unintentionally
justjoehere
commented
1 year ago I think you're encountering a common issue that isn't a flaw exactly. The current chunking approach in this repo splits Text into 1000 characters with 0 overlap by splitting text based on newline characters. If text doesn't have a newline, e.g. a large machine json, image, or text file, the TextSplitting doesn't occur. This then means that the entire doc is sent to OpenAI in one big chunk. Once embedded into tokens, it exceeds the token limit of the API; hence the response you see. See https://github.com/peterw/Chat-with-Github-Repo/blob/main/src/utils/process.py Line 68. You can pass in other splitting characters as an array, e.g. CharacterTextSplitter(separators=["\n\n", "\n", "."], chunk_size=1000, chunk_overlap=0)
You can also switch to a different splitter type - See https://python.langchain.com/en/latest/modules/indexes/text_splitters.html
brianlan
commented
1 year ago I think you're encountering a common issue that isn't a flaw exactly. The current chunking approach in this repo splits Text into 1000 characters with 0 overlap by splitting text based on newline characters. If text doesn't have a newline, e.g. a large machine json, image, or text file, the TextSplitting doesn't occur. This then means that the entire doc is sent to OpenAI in one big chunk. Once embedded into tokens, it exceeds the token limit of the API; hence the response you see. See https://github.com/peterw/Chat-with-Github-Repo/blob/main/src/utils/process.py Line 68. You can pass in other splitting characters as an array, e.g. CharacterTextSplitter(separators=["\n\n", "\n", "."], chunk_size=1000, chunk_overlap=0)
You can also switch to a different splitter type - See https://python.langchain.com/en/latest/modules/indexes/text_splitters.html
Thanks. After I revised the Line 68 to
text_splitter = CharacterTextSplitter(separator="\n", chunk_size=1000, chunk_overlap=0)it worked for me.
hello i am getting this error any idea to fix this :
2023-05-02 16:58:57.051labelgot an empty value. This is discouraged for accessibility reasons and may be disallowed in the future by raising an exception. Please provide a non-empty label and hide it with label_visibility if needed. 2023-05-02 16:59:06.633 Uncaught app exception Traceback (most recent call last): File "/Library/Frameworks/Python.framework/Versions/3.10/lib/python3.10/site-packages/streamlit/runtime/scriptrunner/script_runner.py", line 565, in _run_script exec(code, module.__dict__) File "/Users/xfocus/Downloads/chatRepo/Chat-with-Github-Repo/chat.py", line 104, in <module> output = search_db(user_input) File "/Users/xfocus/Downloads/chatRepo/Chat-with-Github-Repo/chat.py", line 85, in search_db result = qa({"query": query}) File "/Library/Frameworks/Python.framework/Versions/3.10/lib/python3.10/site-packages/langchain/chains/base.py", line 116, in __call__ raise e File "/Library/Frameworks/Python.framework/Versions/3.10/lib/python3.10/site-packages/langchain/chains/base.py", line 113, in __call__ outputs = self._call(inputs) File "/Library/Frameworks/Python.framework/Versions/3.10/lib/python3.10/site-packages/langchain/chains/retrieval_qa/base.py", line 110, in _call answer = self.combine_documents_chain.run( File "/Library/Frameworks/Python.framework/Versions/3.10/lib/python3.10/site-packages/langchain/chains/base.py", line 216, in run return self(kwargs)[self.output_keys[0]] File "/Library/Frameworks/Python.framework/Versions/3.10/lib/python3.10/site-packages/langchain/chains/base.py", line 116, in __call__ raise e File "/Library/Frameworks/Python.framework/Versions/3.10/lib/python3.10/site-packages/langchain/chains/base.py", line 113, in __call__ outputs = self._call(inputs) File "/Library/Frameworks/Python.framework/Versions/3.10/lib/python3.10/site-packages/langchain/chains/combine_documents/base.py", line 75, in _call output, extra_return_dict = self.combine_docs(docs, **other_keys) File "/Library/Frameworks/Python.framework/Versions/3.10/lib/python3.10/site-packages/langchain/chains/combine_documents/stuff.py", line 82, in combine_docs return self.llm_chain.predict(**inputs), {} File "/Library/Frameworks/Python.framework/Versions/3.10/lib/python3.10/site-packages/langchain/chains/llm.py", line 151, in predict return self(kwargs)[self.output_key] File "/Library/Frameworks/Python.framework/Versions/3.10/lib/python3.10/site-packages/langchain/chains/base.py", line 116, in __call__ raise e File "/Library/Frameworks/Python.framework/Versions/3.10/lib/python3.10/site-packages/langchain/chains/base.py", line 113, in __call__ outputs = self._call(inputs) File "/Library/Frameworks/Python.framework/Versions/3.10/lib/python3.10/site-packages/langchain/chains/llm.py", line 57, in _call return self.apply([inputs])[0] File "/Library/Frameworks/Python.framework/Versions/3.10/lib/python3.10/site-packages/langchain/chains/llm.py", line 118, in apply response = self.generate(input_list) File "/Library/Frameworks/Python.framework/Versions/3.10/lib/python3.10/site-packages/langchain/chains/llm.py", line 62, in generate return self.llm.generate_prompt(prompts, stop) File "/Library/Frameworks/Python.framework/Versions/3.10/lib/python3.10/site-packages/langchain/chat_models/base.py", line 82, in generate_prompt raise e File "/Library/Frameworks/Python.framework/Versions/3.10/lib/python3.10/site-packages/langchain/chat_models/base.py", line 79, in generate_prompt output = self.generate(prompt_messages, stop=stop) File "/Library/Frameworks/Python.framework/Versions/3.10/lib/python3.10/site-packages/langchain/chat_models/base.py", line 54, in generate results = [self._generate(m, stop=stop) for m in messages] File "/Library/Frameworks/Python.framework/Versions/3.10/lib/python3.10/site-packages/langchain/chat_models/base.py", line 54, in <listcomp> results = [self._generate(m, stop=stop) for m in messages] File "/Library/Frameworks/Python.framework/Versions/3.10/lib/python3.10/site-packages/langchain/chat_models/openai.py", line 266, in _generate response = self.completion_with_retry(messages=message_dicts, **params) File "/Library/Frameworks/Python.framework/Versions/3.10/lib/python3.10/site-packages/langchain/chat_models/openai.py", line 228, in completion_with_retry return _completion_with_retry(**kwargs) File "/Library/Frameworks/Python.framework/Versions/3.10/lib/python3.10/site-packages/tenacity/__init__.py", line 289, in wrapped_f return self(f, *args, **kw) File "/Library/Frameworks/Python.framework/Versions/3.10/lib/python3.10/site-packages/tenacity/__init__.py", line 379, in __call__ do = self.iter(retry_state=retry_state) File "/Library/Frameworks/Python.framework/Versions/3.10/lib/python3.10/site-packages/tenacity/__init__.py", line 314, in iter return fut.result() File "/Library/Frameworks/Python.framework/Versions/3.10/lib/python3.10/concurrent/futures/_base.py", line 451, in result return self.__get_result() File "/Library/Frameworks/Python.framework/Versions/3.10/lib/python3.10/concurrent/futures/_base.py", line 403, in __get_result raise self._exception File "/Library/Frameworks/Python.framework/Versions/3.10/lib/python3.10/site-packages/tenacity/__init__.py", line 382, in __call__ result = fn(*args, **kwargs) File "/Library/Frameworks/Python.framework/Versions/3.10/lib/python3.10/site-packages/langchain/chat_models/openai.py", line 226, in _completion_with_retry return self.client.create(**kwargs) File "/Library/Frameworks/Python.framework/Versions/3.10/lib/python3.10/site-packages/openai/api_resources/chat_completion.py", line 25, in create return super().create(*args, **kwargs) File "/Library/Frameworks/Python.framework/Versions/3.10/lib/python3.10/site-packages/openai/api_resources/abstract/engine_api_resource.py", line 153, in create response, _, api_key = requestor.request( File "/Library/Frameworks/Python.framework/Versions/3.10/lib/python3.10/site-packages/openai/api_requestor.py", line 226, in request resp, got_stream = self._interpret_response(result, stream) File "/Library/Frameworks/Python.framework/Versions/3.10/lib/python3.10/site-packages/openai/api_requestor.py", line 619, in _interpret_response self._interpret_response_line( File "/Library/Frameworks/Python.framework/Versions/3.10/lib/python3.10/site-packages/openai/api_requestor.py", line 682, in _interpret_response_line raise self.handle_error_response( openai.error.InvalidRequestError: This model's maximum context length is 4097 tokens. However, your messages resulted in 16762 tokens. Please reduce the length of the messages.