search

Maknee / minigpt4.cpp

Port of MiniGPT4 in C++ (4bit, 5bit, 6bit, 8bit, 16bit CPU inference with GGML)
MIT License
549 stars 26 forks source link

Avoid displaying KeyboardInterrupt exception on terminal if user arbitrarily interrupts the inference #10

Closed felladrin closed 11 months ago

felladrin commented 11 months ago

Currently, if the user interrupts the inference via Ctrl/Cmd + C, it displays the following in the terminal:

Traceback (most recent call last):
  File "test.py", line 18, in <module>
    for output in minigpt4_chatbot.generate(
  File "minigpt4_library.py", line 646, in generate
    token = self.library.minigpt4_end_chat(self.ctx, self.n_threads, temp, top_k, top_p, tfs_z, typical_p, repeat_last_n, repeat_penalty, alpha_presence, alpha_frequency, mirostat, mirostat_tau, mirostat_eta, penalize_nl)
  File "minigpt4_library.py", line 419, in minigpt4_end_chat
    self.panic_if_error(self.library.minigpt4_end_chat(ctx.ptr, ctypes.pointer(token), n_threads, temp, top_k, top_p, tfs_z, typical_p, repeat_last_n, repeat_penalty, alpha_presence, alpha_frequency, mirostat, mirostat_tau, mirostat_eta, penalize_nl))
KeyboardInterrupt

Changes on this PR prevent the exception from being displayed if the user arbitrarily interrupts the inference, stopping the inference process cleanly.

Maknee commented 11 months ago

LGTM. Thanks for the catch -- it's pretty common for people to interrupt chat and I've done that before often.