Expand Inline Suggest settings

[marrej]

Problem statement

We get a lot of reports from different user groups that would like to have a better control over what suggestions are shown and where. While some like comment completions, some don't. Some do feel impacted when completions are shown immediately, and some don't like completions shown on specific operations in the editor. Most of these are not solvable in the extension (where we listen to the inline suggest providers), as we do not get enough signals to handle these behaviors.

Proposal

Settings:

• Disable on removal - Stops retrieval of new Inline Suggest completions while user is deleting. In many cases we observed that this would just produce the same line over again to the user, which is not always what users want.
• Disable on location (comment/string) - Same as we have for standard code completion (intellisense) - which would allow for a better control over where should completions render
• Show after n ms of idle - Would allow users to configure when suggestions are shown, to not break their workflows while coding, but e.g. show suggestion only after they have stopped typing for 5 seconds and are stuck.

Nice to have:

It would be great to also add a signal to the Extension that a request has been blocked, delayed, dropped, so FE metrics could reason about why e.g. amount of provided suggestions was reduced.

Alternative:

Add additional signals to the context of the provideInlineCompletion, to allow extensions to make the call whether to add or not to add the filtering. This would remove the need to add additional signals when the calls are dropped on the core end, and would allow extensions to stay in control of the inline completions as well as the logging/metrics.

Happy to help implement if there is no bandwidth and we agree on a way forward :)

[hediet]

I think the feature request is reasonable. Thanks for the detailed write up!

It would be great to also add a signal to the Extension that a request has been blocked, delayed, dropped, so FE metrics could reason about why e.g. amount of provided suggestions was reduced.

How would that signaling look like?

---

I tend to think though that this could be entirely implemented on the extension side. Most implementors already have some kind of syntax tree, so I'm not sure if VS Code should add the requested signals.

Disable on removal [...] In many cases we observed that this would just produce the same line over again to the user, which is not always what users want.

I'm not against the suggestion, but I'm not sure the suggestion is a good fix for problem (that the AI just suggests what the user just deleted and that this is unhelpful). The AI could actually use a deletion as a signal on what not to suggest, or even better pick up the deleted text to predict user intention.

[marrej]

Sorry for the late answer (vacation time in full swing)

How would that signaling look like?

I imagine that we can attach some kind of supplementary additionalInfo on the show/rejection signals (I know that we don't have yet rejection signals, but having those would be very helpful :) Currently we just rely on shown signals with some local patching). based on the specific listeners (model change/ keybinding trigered/ command invoked) we could attach the specified "Event".

I tend to think though that this could be entirely implemented on the extension side. Most implementors already have some kind of syntax tree, so I'm not sure if VS Code should add the requested signals.

I agree that deletion can be detected via the syntax tree or model change, but rejection via esc keys, or invocation via Suggest dropdown invocation is not something that the extension can listen to if I'm not mistaken. So having these signals available would be very helpful.

The model/suggestion provider can potentially use some signals such as force rejecting (via esc) to add a time based snooze, or content based back off to not provide the same completion while users are typing/deleting.

I'm not against the suggestion, but I'm not sure the suggestion is a good fix for problem (that the AI just suggests what the user just deleted and that this is unhelpful). The AI could actually use a deletion as a signal on what not to suggest, or even better pick up the deleted text to predict user intention.

That is a fair point. Would it then possibly make sense to actually fire the request, just allow user to select whether the completion should be shown or not? The check might be fairly straight forward, e.g. if the replacement in the file + current file === last file state before deletion then filter the suggestion. Ofc this might still provide the same response if user goes to a different line adds something and then comes to the same line and starts deleting.