[Bug]: <title>/v1/chat/completions流式的时候无法正常结束

[yc446833448]

Do you need to file an issue?

• [ ] I have searched the existing issues and this bug is not already filed.
• [ ] My model is hosted on OpenAI or Azure. If not, please look at the "model providers" issue and don't file a new one here.
• [ ] I believe this is a legitimate bug, not just a question. If this is a question, please use the Discussions area.

Describe the bug

No response

Steps to reproduce

No response

Expected Behavior

No response

GraphRAG Config Used

# Paste your config here

Logs and screenshots

No response

Additional Information

• GraphRAG Version:
• Operating System:
• Python Version:
• Related Issues:

[yc446833448]

具体在CustomSearchCallback 这个方法里面 while true 结束不了

[yc446833448]

我重写了这块代码，需要什么格式自己改改

class GraphRagLLMCallback(GlobalSearchLLMCallback, BaseLLMCallback):
    """
    Contains functions to define custom callback handlers to enable the streaming of
    graphrag's response via a generator function.
    """

    def __init__(self, token_queue: Queue = None):
        super().__init__()
        self.q = token_queue

    def on_llm_new_token(self, token: str):
        self.q.put(
            json.dumps({"done": False,"token": token, "context": None}).encode("utf-8"), block=True
        )
async def handle_stream_response(request, search, conversation_history):
    future = gtypes.TypedFuture[SearchResult]()

    def stream_response():
        q = Queue()
        callback = GraphRagLLMCallback(token_queue=q)
        # 传递 `callback` 实例给搜索引擎
        search.callbacks = [callback]
        job_done = object()  # signals the processing is done

        async def run_search():
            result = await search.asearch(request.messages[-1].content, conversation_history)
            future.set_result(result)
            q.put(job_done, block=True)  # blocks until the token is consumed by the generator

        # 在一个独立的线程中运行异步任务
        loop = asyncio.new_event_loop()
        asyncio.set_event_loop(loop)
        loop.run_until_complete(run_search())
        loop.close()

        while True:
            next_item = q.get(block=True, timeout=300)  # blocks until an input is available
            if next_item is job_done:
                yield json.dumps({"done": True, "token": "结束", "context": None}).encode("utf-8")
                break
            yield next_item + b"\n"

    return StreamingResponse(stream_response(), media_type="application/x-ndjson")

[KylinMountain]

你是不是copy的官方的graphrag 加我的server？

[yc446833448]

@KylinMountain 我是直接 pip install graphrag 安装的，借鉴了你server的写法。互相学习

[KylinMountain]

奥 那你会遇到无法停止的问题，我改了graphrag 加了end回掉了。那这个在graphrag-server中不是问题