continuedev / continue

⏩ Continue is the leading open-source AI code assistant. You can connect any models and any context to build custom autocomplete and chat experiences inside VS Code and JetBrains

https://docs.continue.dev/

Apache License 2.0

19.43k stars 1.69k forks source link

Flow AbortSignal from IDE #3013

Open X9VoiD opened 1 day ago

X9VoiD commented 1 day ago

Description

2935 had some `AbortController`s with no way to call `abort()` on it. This PR fixes that and only adds AbortSignal where it makes sense. For future proofing, AbortSignal is also passed down to `core/llm/stream.ts` functions.

The abort error that shows up in debug console is also removed.

Testing

Tested with ollama serve. No requests are left open. Cached completions still work. No regressions are detected so far with manual testing.

netlify[bot] commented 1 day ago

Deploy Preview for continuedev failed. Why did it fail? →

Name	Link
Latest commit	d1551627b8ca846a319583ea6885ab40edf67311
Latest deploy log	https://app.netlify.com/sites/continuedev/deploys/673f78bbc178620008e828eb

fbricon commented 2 hours ago

Binding the http requests to the IDE cancellation token was the 1st approach I used in #2935. But as @sestinj requested, and as https://github.com/continuedev/continue/pull/2935#issuecomment-2488927707 explains, the abort signal has to be controlled by the generator, so you can still reuse pending streaming if the prefix is compatible with previous completion requests, as to minimize requests. This is irrelevant to response caching, that you observed.