use streaming RPC API exthost<->webview using AsyncGenerator

[sqs]

Now, the webview can call into API methods exposed by the ChatController (extension host), and those API methods can return a stream of values. This lets the webview "observe" things in the extension host.

Fixes several issues:

• OpenCtx @-mention providers now load immediately when each is ready, without waiting for all to be ready. We had a 5-second hardcoded delay for providers, which meant that in many cases the first chat session would not have any OpenCtx providers. Also, if you change your configuration, the new providers appear immediately in all chats.
• Feature flags used in the webview with useFeatureFlag now load properly. This fixes an issue where the Prompts dropdown was not displaying, even for users for whom it was enabled.
• Context items now stream in, which means that an OpenCtx provider can emit multiple values as it loads more and more data.
• Webview<->exthost API calls are now aborted automatically (when using the useAsyncGenerator React hook).

Test plan

CI. Also a lot of manual usage of @-mentions and the Prompts dropdown, since those are the parts that use this new API.

[sqs]

@dominiccooney Thanks, I just addressed your comments and re-pushed.

[sqs]

@dominiccooney OK to merge and do a post-merge review? This will fix the feature flag listener problem and unblock more testing of the prompt library prior to the release.

[dominiccooney]

@sqs review post merge sounds good to me. We should work out with search team who is on point for bigger changes like this so they can really own the result.

[vovakulikov]

Before merging this can someone check Cody web demo. I can check it later today myself but maybe it would be too late since this has been approved already

[sqs]

@vovakulikov I’ve checked it and it works