API: Using `sendRequest` and getting cut off response (including in chat-sample)

[worksofliam]

My Copilot responses are being cut of when using the API to implement my chat. It doesn't seem to happen all the time.

Code

```ts async function streamModelResponse( messages: vscode.LanguageModelChatMessage[], stream: vscode.ChatResponseStream, token: vscode.CancellationToken ) { const chosenProvider = AiConfig.getProvider(); const chosenModel = AiConfig.getModel(); if (chosenProvider === `none`) { stream.markdown( `No AI provider selected. Please select an AI provider and model.` ); stream.button({ command: `vscode-db2i.ai.changeModel`, title: `Select AI Provider and Model`, }); return; } showModelProviderIfNeeded(stream, chosenProvider, chosenModel); stream.progress(`Provider: ${chosenProvider} Model: ${chosenModel}`); return copilotRequest(chosenModel, messages, {}, token, stream); } async function copilotRequest( model: string, messages: LanguageModelChatMessage[], options: LanguageModelChatRequestOptions, token: CancellationToken, stream: vscode.ChatResponseStream ): Promise { const models = await vscode.lm.selectChatModels({ family: model }); if (models.length > 0) { const [first] = models; const response = await first.sendRequest(messages, options, token); try { for await (const fragment of response.text) { stream.markdown(fragment); } } catch (e) { console.error(e); stream.markdown("An error occurred while processing the request."); } } } ```

More details with screenshots

When using standard copilot, it does not cut off. Could it be the case that my input/context is too big?  Further progress using a different model but same issue.  Took out all my User messages / context and still getting the same issue. 

With logs:

2024-07-01 14:49:23.623 [info] [chat fetch] request.response: [https://api.githubcopilot.com/chat/completions], took 855 ms
2024-07-01 14:49:26.248 [info] [streamMessages] message 0 returned. finish reason: [stop]
2024-07-01 14:49:26.249 [info] [streamChoices] request done: requestId: [2495000d-a584-4652-80f2-a4b1af36754c] model deployment ID: []

[worksofliam]

I have a colleague who is checked out with the same code and the issue is not happening for them.

[worksofliam]

More requests that finish but seemingly cut off.

Screenshot

2024-07-01 15:51:47.308 [info] [chat fetch] request.response: [https://api.githubcopilot.com/chat/completions], took 1089 ms
2024-07-01 15:52:02.460 [info] [streamMessages] message 0 returned. finish reason: [stop]
2024-07-01 15:52:02.461 [info] [streamChoices] request done: requestId: [2f06e0be-77a4-40b3-a539-f794b69e916e] model deployment ID: []
2024-07-01 15:53:31.062 [info] [chat fetch] request.response: [https://api.githubcopilot.com/chat/completions], took 1203 ms
2024-07-01 15:53:34.183 [info] [streamMessages] message 0 returned. finish reason: [stop]
2024-07-01 15:53:34.184 [info] [streamChoices] request done: requestId: [9e77cd62-0045-4818-9f29-26f48f0aa608] model deployment ID: []

[worksofliam]

I have recreate this issue with the vscode-extension-samples -> chat-sample: https://github.com/microsoft/vscode-extension-samples/tree/main/chat-sample

2024-07-01 16:00:55.412 [info] [chat fetch] request.response: [https://api.githubcopilot.com/chat/completions], took 329 ms
2024-07-01 16:00:55.929 [info] [streamMessages] message 0 returned. finish reason: [stop]
2024-07-01 16:00:55.929 [info] [streamChoices] request done: requestId: [94db1a75-8a4f-4dba-814c-6fddb30bc239] model deployment ID: [dep-3]

Copilot Diags

## Copilot - Version: 1.208.962 - Build: nightly - Editor: vscode/1.91.0-insider ## Environment - http_proxy: n/a - https_proxy: n/a - no_proxy: n/a - SSL_CERT_FILE: n/a - SSL_CERT_DIR: n/a - OPENSSL_CONF: n/a ## Feature Flags - Send Restricted Telemetry: enabled - Chat: enabled ## Node setup - Number of root certificates: 141 - Operating system: Darwin - Operating system version: 23.5.0 - Operating system architecture: arm64 - NODE_OPTIONS: n/a - NODE_EXTRA_CA_CERTS: n/a - NODE_TLS_REJECT_UNAUTHORIZED: n/a - tls default min version: TLSv1.2 - tls default max version: TLSv1.3 ## Network Configuration - Proxy host: n/a - Proxy port: n/a - Kerberos SPN: n/a - Reject unauthorized: disabled - Fetcher: HelixFetcher ## Reachability - github.com: HTTP 200 - api.github.com: HTTP 200 - copilot-proxy.githubusercontent.com: HTTP 200 - api.githubcopilot.com: HTTP 200 - default.exp-tas.com: HTTP 200 ## VS Code Configuration - HTTP proxy: - HTTP proxy authentication: n/a - Proxy Strict SSL: true - Extension HTTP proxy support: override ## Extensions - Is `win-ca` installed?: false - Is `mac-ca` installed?: false ## Authentication - GitHub username: worksofliam

Screenshot showing normal Copilot works fine

[roblourens]

So you're seeing the issue with the chat-sample, and no modifications to it? A couple days later, is this still happening? Could have been a transient service issue.

It looks like you found the "Copilot: Collect Diagnostics" command, thanks. Could you also run "Developer: GitHub Copilot Chat Diagnostics"?

I'm not sure what else to look at, any ideas @jrieken?

[jrieken]

What version of VS Code and chat do you use? Around a week ago there was a bug in this area which since has been fixed