[BUG] chunk1['attention_mask'].append(chunk2['attention_mask'][0]) IndexError: list index out of range

[GitHubHaoP]

是否已有关于该错误的issue或讨论？ | Is there an existing issue / discussion for this?

• [X] 我已经搜索过已有的issues和讨论 | I have searched the existing issues / discussions

该问题是否在FAQ中有解答？ | Is there an existing answer for this in FAQ?

• [X] 我已经搜索过FAQ | I have searched FAQ

当前行为 | Current Behavior

[2024-05-16 17:21:40 +0800] [39846] [ERROR] Exception occurred while handling uri: 'http://127.0.0.1:8777/api/local_doc_qa/local_doc_chat' Traceback (most recent call last): File "handle_request", line 132, in handle_request "_asgi_lifespan", File "/root/miniconda3/envs/qanything-python/lib/python3.10/site-packages/sanic/response/types.py", line 547, in stream await self.streaming_fn(self) File "/home/haopeng/QAnything/qanything_kernel/qanything_server/handler.py", line 398, in generate_answer async for resp, next_history in local_doc_qa.get_knowledge_based_answer(custom_prompt=custom_prompt, File "/home/haopeng/QAnything/qanything_kernel/core/local_doc_qa.py", line 262, in get_knowledge_based_answer retrieval_documents = await self.retrieve(query, kb_ids, need_web_search=need_web_search) File "/home/haopeng/QAnything/qanything_kernel/core/local_doc_qa.py", line 246, in retrieve retrieval_documents = await self.local_doc_search(query, kb_ids) File "/home/haopeng/QAnything/qanything_kernel/core/local_doc_qa.py", line 128, in local_doc_search retrieval_documents = self.rerank_documents(query, retrieval_documents) File "/home/haopeng/QAnything/qanything_kernel/core/local_doc_qa.py", line 238, in rerank_documents scores = self.local_rerank_backend.predict(query, [doc.page_content for doc in source_documents]) File "/home/haopeng/QAnything/qanything_kernel/utils/general_utils.py", line 72, in inner res = func(*arg, **kwargs) File "/home/haopeng/QAnything/qanything_kernel/connector/rerank/rerank_backend.py", line 92, in predict tot_batches, merge_inputs_idxs_sort = self.tokenize_preproc(query, passages) File "/home/haopeng/QAnything/qanything_kernel/connector/rerank/rerank_backend.py", line 71, in tokenize_preproc qp_merge_inputs = self.merge_inputs(query_inputs, passage_inputs) File "/home/haopeng/QAnything/qanything_kernel/connector/rerank/rerank_backend.py", line 47, in merge_inputs chunk1['attention_mask'].append(chunk2['attention_mask'][0]) IndexError: list index out of range

期望行为 | Expected Behavior

能根据知识库中上传的文档，正确回答

运行环境 | Environment

- OS:
- NVIDIA Driver:
- CUDA:
- docker:
- docker-compose:
- NVIDIA GPU:
- NVIDIA GPU Memory:

QAnything日志 | QAnything logs

No response

复现方法 | Steps To Reproduce

No response

备注 | Anything else?

No response

[a614349007]

大佬解决了嘛？我这里也是有这个问题