search

ahyatt / llm

A package abstracting llm capabilities for emacs.
GNU General Public License v3.0
142 stars 19 forks source link

Return request buffer from `llm-request-async' #11

Closed Stebalien closed 6 months ago

Stebalien commented 6 months ago

Return request buffer from llm-request-async, allowing the user can cancel the request by killing the buffer. This will, in turn, kill the and cancel HTTP request.

I've tested this with Ollama and it appears to kill the underlying request as well.

fixes #6

ahyatt commented 6 months ago

Thanks for the fix. This is part of a fix for #6, but not sufficient. I have the other part written already, which is supporting a cancel-query method in the llm library. This would be nice in the future if there are any llms that connect in a way that aren't buffer-based. There's also another problem - I've found that killing the buffer often will cause the url request library to throw an error. I've had to wrap killing the buffer with (cl-letf (((symbol-function 'url-http-async-sentiel) (lambda (_ _)))) (kill-buffer)).

So I don't mind merging this in, but you can also wait until my more complete fix is in, which I think is necessary. Sorry for the delay, though, it's been taking longer than I've though.

Stebalien commented 6 months ago

Ah, I see. This does seem to be sufficient for ollama as canceling the HTTP request cancels the rest of the request as well.

If you want to support LLMs that connect without buffers, it's probably best not to merge this (lest users depend on that behavior).

So I don't mind merging this in, but you can also wait until my more complete fix is in, which I think is necessary. Sorry for the delay, though, it's been taking longer than I've though.

No problem, I completely understand.