search

abetlen / llama-cpp-python

Python bindings for llama.cpp
https://llama-cpp-python.readthedocs.io
MIT License
8.16k stars 970 forks source link

fix: added missing exit_stack.close() to /v1/chat/completions #1796

Open Ian321 opened 1 month ago

Ian321 commented 1 month ago

server.create_chat_completion would not close the exit_stack if an exception happened while calling llama.create_chat_completion.

This PR fixes #1759 and is a superset of #1795.

nathan-weinberg commented 1 month ago

+1 to this fix - current we are blocked in bumping our version of llama-cpp-python due to this bug