QiongsK
commented
11 months ago 我也遇到了这个问题
Closed 202030481266 closed 11 months ago
QiongsK
commented
11 months ago 我也遇到了这个问题
leilichuan
commented
11 months ago 包的版本问题，我开始用的qwen-14B-chat也报这个错，后来降低了torch的版本。 requirements.txt里面限制torch==2.0.1 langchain>=0.0.319 langchain-experimental # >=0.0.30 fschat[model_worker] #==0.2.31 xformers #>=0.0.22.post4 openai>=0.28.1 sentence_transformers transformers #>=4.34 torch==2.0.1 # 推荐2.1 torchvision torchaudio fastapi>=0.104 nltk>=3.8.1 uvicorn~=0.23.1 starlette~=0.27.0 pydantic~=1.10.11 unstructured[all-docs]>=0.10.12 python-magic-bin; sys_platform == 'win32' SQLAlchemy==2.0.19 faiss-cpu accelerate spacy PyMuPDF rapidocr_onnxruntime
202030481266
commented
11 months ago 似乎并不是这个库的版本问题，因为我现在的环境就是：
hzg0601
commented
11 months ago 可调用openai api界面，通过 直接与大模型对话 来定位真正的问题
202030481266
commented
11 months ago 经过测试应该是分词器的问题，测试了一下知识库的匹配条数并不重要，重要的是加起来的上下文长度，实测GPT3的分词器下一万多的token是可以的。
202030481266
commented
11 months ago @hzg0601 对的，在很长的上下文对话中，直接与LLM对话也会报这个错误。
leoterry-ulrica
commented
11 months ago 加载baichuan2-13b-chat-8bit量化模型，对话的时候提示同样的错误，得不到答案。
202030481266
commented
11 months ago @leoterry-ulrica 这个问题发生在流式生成，因此任何可以引发模型错误的内容都会返回这个错误，因为langchain的openai.py内部采用了不安全的写法： len['choice']，因此可以将这个看作模型不正常运行。比如我的就是输入文本太长了才导致了模型不能正确运行。
roki1031
commented
11 months ago 参考https://github.com/chatchat-space/Langchain-Chatchat/issues/1840 可解决
@pdo2013 requirements.txt和requirements.txt_api.txt 按以上设置Qwen可以访问。谢谢！
Donovan-Ye
commented
8 months ago 也可以看一下gpu显存是不是够用，知识库筛选出条数多的时候，会因为显存不够而报这个错。。我们这边换了一台机器就可以了
qianbaidu1266
commented
8 months ago 降低torch 版本后，会报新的错误： utils.py[line:29] - ERROR: TypeError: Caught exception: object of type 'NoneType' has no len()
问题描述 / Problem Description 使用了chinese-llama-alpaca-2-16k作为LLM聊天模型，在知识库问答模式下，回答失效，只返回了知识库的匹配内容。主要原因在于返回的匹配内容长度问题，如果选择只返回一条匹配的内容，模型可以正常作答，但是如果返回两条及以上就会不正常。
2023-10-28 12:48:15 | INFO | httpx | HTTP Request: POST http://127.0.0.1:20002/worker_generate_stream "HTTP/1.1 200 OK" {'docs': [...]} 2023-10-28 12:48:16,421 - utils.py[line:26] - ERROR: KeyError: Caught exception: 'choices' Traceback (most recent call last): File "/root/autodl-tmp/Langchain-Chatchat/server/utils.py", line 22, in wrap_done await fn File "/root/miniconda3/envs/llm/lib/python3.10/site-packages/langchain/chains/base.py", line 379, in acall raise e File "/root/miniconda3/envs/llm/lib/python3.10/site-packages/langchain/chains/base.py", line 373, in acall await self._acall(inputs, run_manager=run_manager) File "/root/miniconda3/envs/llm/lib/python3.10/site-packages/langchain/chains/llm.py", line 239, in _acall response = await self.agenerate([inputs], run_manager=run_manager) File "/root/miniconda3/envs/llm/lib/python3.10/site-packages/langchain/chains/llm.py", line 117, in agenerate return await self.llm.agenerate_prompt( File "/root/miniconda3/envs/llm/lib/python3.10/site-packages/langchain/chat_models/base.py", line 469, in agenerate_prompt return await self.agenerate( File "/root/miniconda3/envs/llm/lib/python3.10/site-packages/langchain/chat_models/base.py", line 429, in agenerate raise exceptions[0] File "/root/miniconda3/envs/llm/lib/python3.10/site-packages/langchain/chat_models/base.py", line 532, in _agenerate_with_cache return await self._agenerate( File "/root/miniconda3/envs/llm/lib/python3.10/site-packages/langchain/chat_models/openai.py", line 433, in _agenerate return await _agenerate_from_stream(stream_iter) File "/root/miniconda3/envs/llm/lib/python3.10/site-packages/langchain/chat_models/base.py", line 70, in _agenerate_from_stream async for chunk in stream: File "/root/miniconda3/envs/llm/lib/python3.10/site-packages/langchain/chat_models/openai.py", line 405, in _astream if len(chunk["choices"]) == 0: KeyError: 'choices'
复现问题的步骤 / Steps to Reproduce
环境信息 / Environment Information
附加信息 / Additional Information 可以确定问题发生在模型的流式生成上。