Open marrej opened 1 month ago
I think the feature request is reasonable. Thanks for the detailed write up!
It would be great to also add a signal to the Extension that a request has been blocked, delayed, dropped, so FE metrics could reason about why e.g. amount of provided suggestions was reduced.
How would that signaling look like?
I tend to think though that this could be entirely implemented on the extension side. Most implementors already have some kind of syntax tree, so I'm not sure if VS Code should add the requested signals.
Disable on removal [...] In many cases we observed that this would just produce the same line over again to the user, which is not always what users want.
I'm not against the suggestion, but I'm not sure the suggestion is a good fix for problem (that the AI just suggests what the user just deleted and that this is unhelpful). The AI could actually use a deletion as a signal on what not to suggest, or even better pick up the deleted text to predict user intention.
Sorry for the late answer (vacation time in full swing)
How would that signaling look like?
I imagine that we can attach some kind of supplementary additionalInfo
on the show/rejection signals (I know that we don't have yet rejection signals, but having those would be very helpful :) Currently we just rely on shown signals with some local patching). based on the specific listeners (model change/ keybinding trigered/ command invoked) we could attach the specified "Event".
I tend to think though that this could be entirely implemented on the extension side. Most implementors already have some kind of syntax tree, so I'm not sure if VS Code should add the requested signals.
I agree that deletion can be detected via the syntax tree or model change, but rejection via esc keys, or invocation via Suggest dropdown invocation is not something that the extension can listen to if I'm not mistaken. So having these signals available would be very helpful.
The model/suggestion provider can potentially use some signals such as force rejecting (via esc) to add a time based snooze, or content based back off to not provide the same completion while users are typing/deleting.
I'm not against the suggestion, but I'm not sure the suggestion is a good fix for problem (that the AI just suggests what the user just deleted and that this is unhelpful). The AI could actually use a deletion as a signal on what not to suggest, or even better pick up the deleted text to predict user intention.
That is a fair point. Would it then possibly make sense to actually fire the request, just allow user to select whether the completion should be shown or not?
The check might be fairly straight forward, e.g. if the replacement in the file + current file === last file state before deletion
then filter the suggestion. Ofc this might still provide the same response if user goes to a different line adds something and then comes to the same line and starts deleting.
Problem statement
We get a lot of reports from different user groups that would like to have a better control over what suggestions are shown and where. While some like comment completions, some don't. Some do feel impacted when completions are shown immediately, and some don't like completions shown on specific operations in the editor. Most of these are not solvable in the extension (where we listen to the inline suggest providers), as we do not get enough signals to handle these behaviors.
Proposal
Settings:
Nice to have:
It would be great to also add a signal to the Extension that a request has been blocked, delayed, dropped, so FE metrics could reason about why e.g. amount of provided suggestions was reduced.
Alternative:
Add additional signals to the context of the
provideInlineCompletion
, to allow extensions to make the call whether to add or not to add the filtering. This would remove the need to add additional signals when the calls are dropped on the core end, and would allow extensions to stay in control of the inline completions as well as the logging/metrics.Happy to help implement if there is no bandwidth and we agree on a way forward :)