search

leona / helix-gpt

Code assistant language server for Helix with support for Copilot/OpenAI/Codeium/Ollama
MIT License
285 stars 19 forks source link

[BUG] Completion-requests to Ollama are not cancelled, resulting in an unusable delay #53

Open justinlovinger opened 2 months ago

justinlovinger commented 2 months ago

helix-editor version helix 23.10 (f6021dd0)

helix-gpt version 0.31

Describe the bug Ollama supports canceling requests, https://github.com/ollama/ollama/issues/2876#issuecomment-1989771389. However, from my testing of using Ollama from helix-gpt, code-completion requests are not cancelled. That means every few times a character is typed a new request is queued. The latest request will never finish in a reasonable time.

sigmaSd commented 2 months ago

@justinlovinger can you test this branch https://github.com/leona/helix-gpt/compare/master...sigmaSd:helix-gpt:perf

if that works , I can open a pr

justinlovinger commented 2 months ago

@justinlovinger can you test this branch master...sigmaSd:helix-gpt:perf

if that works , I can open a pr

Your change appears to work. Ollama is more responsive, and I was able to get completions.

Something still felt off. I got completions less often than I expected to. However, that may have been caused by the model I was using.

Also, Helix would frequently show "The operation was aborted." as an error, but that is a nitpick.

sigmaSd commented 2 months ago

Also, Helix would frequently show "The operation was aborted." as an error, but that is a nitpick.

try adding this line if (error.name === "AbortError") reject({ message: "" }); here https://github.com/sigmaSd/helix-gpt/blob/34f9a7b615d278eba7fc37352a2536b01357476e/src/models/api.ts#L45 (before the other reject) to hide that error

justinlovinger commented 2 months ago

Also, Helix would frequently show "The operation was aborted." as an error, but that is a nitpick.

try adding this line if (error.name === "AbortError") reject({ message: "" }); here https://github.com/sigmaSd/helix-gpt/blob/34f9a7b615d278eba7fc37352a2536b01357476e/src/models/api.ts#L45 (before the other reject) to hide that error

That appears to work.

P.S. I think I figured out why completion felt off, and I opened a separate issue about it, https://github.com/leona/helix-gpt/issues/56.

justinlovinger commented 2 months ago

Another minor issue, requests are not cancelled when timed out.

sigmaSd commented 2 months ago

I added canceling request on timeout here https://github.com/leona/helix-gpt/compare/master...sigmaSd:helix-gpt:perf

justinlovinger commented 2 months ago

I added canceling request on timeout here master...sigmaSd:helix-gpt:perf

It worked for me, 👍.