Adds SmartThrottleService which we implemented before, but removed as it showed negative results.
Makes a minor improvement to reduce the debounce time to 0 when the 250ms throttle timeout is bypassed
Fixes the issue we didn't notice before with SmartThrottleService:
We propagated the abort signal straight to the request manager with this change, which meant the SmartThrottleService could handle aborting requests as new ones came in. This was sound in theory, but we ran into problems because we also listened for cancellationToken.onCancellationRequested to abort the request. This meant that actually VS Code was aborting our requests, rather than the smart throttle, and we only ever had 1 inflight request at a time.
Removes old, unused feature flags that had similar logic:
CodyAutocompleteEagerCancellation
CodyAutocompleteReducedDebounce
Test plan
Tested locally with lots of logging on # of inflight requests, request cancellation and request start
Description
This PR:
SmartThrottleServicewhich we implemented before, but removed as it showed negative results.0when the 250ms throttle timeout is bypassedSmartThrottleService:SmartThrottleServicecould handle aborting requests as new ones came in. This was sound in theory, but we ran into problems because we also listened forcancellationToken.onCancellationRequestedto abort the request. This meant that actually VS Code was aborting our requests, rather than the smart throttle, and we only ever had 1 inflight request at a time.CodyAutocompleteEagerCancellationCodyAutocompleteReducedDebounceTest plan