Copilot Chat confabulation: `"github.copilot.editor.enableTypeAnnotations": true

[bartoncreek]

This is to report Copilot Chat's confabulation as to its own settings. All information is below.

Chat transcript

bartoncreek: /vscode is it possible to set some default instructions for copilot that specify codpilot's generated python code include type annotations whenever possible

GitHub Copilot: Yes, it is possible to set default instructions for Copilot to include type annotations whenever possible. You can enable this feature by setting the following setting in your VS Code settings.json file:

{
    "github.copilot.editor.enableTypeAnnotations": true
}

This will instruct Copilot to generate Python code with type annotations whenever possible.

bartoncreek: Can you doublecheck the accuracy of this setting. Does it really exist?

GitHub Copilot: I apologize for the confusion. The setting I mentioned earlier, "github.copilot.editor.enableTypeAnnotations", does not actually exist. I made a mistake in my previous response. However, Copilot does generate type annotations in Python code when it is able to infer the types. So, while there is no setting to enable this behavior, Copilot will automatically include type annotations when it can.

My System and Versions

• Copilot Chat ext version: v0.7.2023080
• Copilot extension version: v1.101.317
• VS Code Version: 1.81.0
• Date: 2023-08-02T12:36:11.334Z
• Electron: 22.3.18
• ElectronBuildId: 22689846
• Chromium: 108.0.5359.215
• Node.js: 16.17.1
• V8: 10.8.168.25-electron.0
• OS: Linux x64 6.4.4-arch1-1

Steps to Reproduce:

1. Well, given the well-known stochastic nature of LLM confabulations, this is just a "ONE-SHOT" report for the experts to suss out.

Chat Screenshot

[KalleV]

I was able to replicate this report in a slightly different way using the /vscode shortcut:

[bartoncreek]

I'm bored, so I'll venture a suggestion for the devs for possible anti-confabulation measures. In short, /vscode chats about settings must trigger a secondary "Recursive Criticism" check before spitting out one–few shot response.

See Language Models can Solve Computer Tasks