search

netease-youdao / QAnything

Question and Answer based on Anything.
https://qanything.ai
GNU Affero General Public License v3.0
11.4k stars 1.1k forks source link

[BUG] 目前遇到的几个问题 #510

Open gotothehill opened 1 week ago

gotothehill commented 1 week ago

是否已有关于该错误的issue或讨论？ | Is there an existing issue / discussion for this?

该问题是否在FAQ中有解答？ | Is there an existing answer for this in FAQ?

当前行为 | Current Behavior

  1. api页面显示404
  2. milvus解析pdf时大概率失败，一般是两种错误情况：milvus insert timeout: 300s，milvus insert error。
  3. 创建bot之后，预览体验，提示：fail, api_base is required

期望行为 | Expected Behavior

  1. api 页面能正常访问
  2. 超时是否只需要修改milvus配置就可以，配置路径（Windows docker下），怎么修改。pdf有没有什么正确的上传方式，预处理什么的，milvus insert error错误日志在下方
  3. bot的正确配置使用方法，我配置的大模型是本地运行的 ollama qwen2:72b

运行环境 | Environment

- OS: Windows 11 WSL2
- NVIDIA Driver: 560.81 
- CUDA: 12.6
- Docker Compose: 4.33.1
- NVIDIA GPU Memory: 24GB x 4

QAnything日志 | QAnything logs

milvus insert error 日志：

Aco...

2024-09-11 02:09:25,573 - [PID: 1384][Sanic-Server-0-0] - [Function: aadd_texts] - ERROR - Failed to insert batch starting at entity: 0/510 2024-09-11 02:09:25,608 - [PID: 1384][Sanic-Server-0-0] - [Function: process_data] - ERROR - milvus insert error: Traceback (most recent call last): File "/workspace/QAnything/qanything_kernel/dependent_server/insert_files_serve/insert_files_server.py", line 108, in process_data chunks_number, insert_time_record = await asyncio.wait_for( File "/usr/local/lib/python3.10/asyncio/tasks.py", line 445, in wait_for return fut.result() File "/workspace/QAnything/qanything_kernel/utils/general_utils.py", line 166, in get_time_async_inner res = await func(args, kwargs) # 注意这里使用 await 来调用异步函数 File "/workspace/QAnything/qanything_kernel/core/retriever/parent_retriever.py", line 210, in insert_documents return await self.retriever.aadd_documents(docs, parent_chunk_size=parent_chunk_size, File "/workspace/QAnything/qanything_kernel/core/retriever/parent_retriever.py", line 141, in aadd_documents res = await self.vectorstore.aadd_documents(embed_docs, time_record=time_record) File "/usr/local/lib/python3.10/site-packages/langchain_core/vectorstores.py", line 153, in aadd_documents return await self.aadd_texts(texts, metadatas, kwargs) File "/workspace/QAnything/qanything_kernel/core/retriever/vectorstore.py", line 252, in aadd_texts raise e File "/workspace/QAnything/qanything_kernel/core/retriever/vectorstore.py", line 242, in aadd_texts res: MutationResult = await asyncio.to_thread( File "/usr/local/lib/python3.10/asyncio/threads.py", line 25, in to_thread return await loop.run_in_executor(None, func_call) File "/usr/local/lib/python3.10/concurrent/futures/thread.py", line 58, in run result = self.fn(self.args, self.kwargs) File "/usr/local/lib/python3.10/site-packages/pymilvus/orm/collection.py", line 497, in insert res = conn.batch_insert( File "/usr/local/lib/python3.10/site-packages/pymilvus/decorators.py", line 135, in handler raise e from e File "/usr/local/lib/python3.10/site-packages/pymilvus/decorators.py", line 131, in handler return func(*args, *kwargs) File "/usr/local/lib/python3.10/site-packages/pymilvus/decorators.py", line 170, in handler return func(self, args, kwargs) File "/usr/local/lib/python3.10/site-packages/pymilvus/decorators.py", line 110, in handler raise e from e File "/usr/local/lib/python3.10/site-packages/pymilvus/decorators.py", line 74, in handler return func(*args, **kwargs) File "/usr/local/lib/python3.10/site-packages/pymilvus/client/grpc_handler.py", line 566, in batch_insert raise err from err File "/usr/local/lib/python3.10/site-packages/pymilvus/client/grpc_handler.py", line 549, in batch_insert request = self._prepare_batch_insert_request( File "/usr/local/lib/python3.10/site-packages/pymilvus/client/grpc_handler.py", line 533, in _prepare_batch_insert_request else Prepare.batch_insert_param(collection_name, entities, partition_name, fields_info) File "/usr/local/lib/python3.10/site-packages/pymilvus/client/prepare.py", line 521, in batch_insert_param location = cls._pre_batch_check(entities, fields_info) File "/usr/local/lib/python3.10/site-packages/pymilvus/client/prepare.py", line 466, in _pre_batch_check location, primary_key_loc, auto_id_loc = traverse_info(fields_info, entities) File "/usr/local/lib/python3.10/site-packages/pymilvus/client/utils.py", line 270, in traverse_info raise ParamError( pymilvus.exceptions.ParamError: <ParamError: (code=1, message=Collection field dim is 768, but entities field dim is 11)>

2024-09-11 02:09:25,621 - [PID: 1384][Sanic-Server-0-0] - [Function: check_and_process] - INFO - time_record: {"parse_time": 23.88, "insert_error": true}

复现方法 | Steps To Reproduce

  1. 下载项目： git clone https://github.com/netease-youdao/QAnything.git
  2. docker启动：docker compose -f docker-compose-win.yaml up
  3. 打开api页面，显示404
  4. 打开前端页面，上传docs基本都能解析成功，上传pdf文档，大概率解析失败：milvus insert timeout，milvus insert error
  5. 创建机器人，创建页面正常使用，发布预览报错：fail, api_base is required

备注 | Anything else?

image api页面

image bot发布预览

MI-Class commented 1 week ago

bot这个我看最新的代码已经修复了

jacktpy commented 1 week ago

我都启动不了。一直等待启动后端服务。。

gotothehill commented 1 week ago

bot这个我看最新的代码已经修复了

看到了，bot分享可以使用了，感谢！

gotothehill commented 1 week ago

我都启动不了。一直等待启动后端服务。。

我只是按官方的步骤启动的，clone仓库，docker启动。我是在Windows环境下的。 这次更新仓库后，scripts/entrypoint.sh在我这显示是使用的Windows的换行符，不能启动，我把这个脚本换成Unix换行符后就正常启动了。

qazxswaaa commented 1 week ago

有人能在windows下启动后上传文件解析成功吗

jacktpy commented 6 days ago

有人能在windows下启动后上传文件解析成功吗

我用百度网盘下的镜像启动成功了一次，但是上传文件解析不了，一直排队中。后来关了重启就又不行了。

MI-Class commented 6 days ago

有人能在windows下启动后上传文件解析成功吗

我用百度网盘下的镜像启动成功了一次，但是上传文件解析不了，一直排队中。后来关了重启就又不行了。

解析不了尝试修改下qanything_kernel/dependent_server/insert_files_serve/insert_files_server.py文件里把setup_workers方法里的app.add_task修改为asyncio.create_task