Supports for o1 models?

[alwint3r]

GitHub has opened a limited preview for OpenAI o1 models. I was lucky enough to be admitted into their limited preview program, and I can see the o1 and o1-preview models when running the :CopilotChatModels command.

Select a model
1: gpt-3.5-turbo-0613
2: gpt-4-0613
3: gpt-4o-2024-05-13
4: gpt-4o-mini-2024-07-18
5: gpt-4-0125-preview
6: o1-preview-2024-09-12
7: o1-mini-2024-09-1

I tried selecting the o1-mini-2024-09-1 model and asked it some questions but got some errors. Here's some details from the log file

[DEBUG Tue Sep 24 23:26:58 2024] CopilotChat.nvim/lua/CopilotChat/copilot.lua:364: Prompt: Why do we need memory safety at all?
[DEBUG Tue Sep 24 23:26:58 2024] CopilotChat.nvim/lua/CopilotChat/copilot.lua:365: Embeddings: 0
[DEBUG Tue Sep 24 23:26:58 2024 CopilotChat.nvim/lua/CopilotChat/copilot.lua:366: Filename: /Users/alwin/copilot-chat
[DEBUG Tue Sep 24 23:26:58 2024] CopilotChat.nvim/lua/CopilotChat/copilot.lua:367: Filetype: copilot-chat
[DEBUG Tue Sep 24 23:26:58 2024] CopilotChat.nvim/lua/CopilotChat/copilot.lua:368: Selection:
[DEBUG Tue Sep 24 23:26:58 2024] CopilotChat.nvim/lua/CopilotChat/copilot.lua:369: Model: o1-mini-2024-09-12
[DEBUG Tue Sep 24 23:26:58 2024] CopilotChat.nvim/lua/CopilotChat/copilot.lua:370: Temperature: 0.1
[ERROR Tue Sep 24 23:27:00 2024] CopilotChat.nvim/lua/CopilotChat/copilot.lua:481: Failed parse response:
Bad Request
"Expected value but found invalid token at character 1"

[gptlang]

I haven't got access yet so I'm not sure about the request/response structure. If someone could send a MITM capture, I'd be able to implement it.

[alwint3r]

I think I have the time to capture that. I'll see what I can do.

[alwint3r]

Here's the raw request and response when asking o1-preview model in the VSCode Insider

POST https://api.githubcopilot.com/chat/completions HTTP/2.0
content-length: 2566
content-type: application/json
copilot-integration-id: vscode-chat
editor-plugin-version: copilot-chat/0.21.2024092501
editor-version: vscode/1.94.0-insider
openai-intent: conversation-panel
openai-organization: github-copilot
user-agent: GitHubCopilotChat/0.21.2024092501
x-github-api-version: 2023-07-07
sec-fetch-site: none
sec-fetch-mode: no-cors
sec-fetch-dest: empty
accept-encoding: gzip, deflate, br
priority: u=4, i

{"messages":[{"role":"user","content":"You are an AI programming assistant.\nWhen asked for your name, you must respond with \"GitHub Copilot\".\nFollow the user's requirements carefully & to the letter.\nFollow Microsoft content policies.\nAvoid content that violates copyrights.\nIf you are asked to generate content that is harmful, hateful, racist, sexist, lewd, violent, or completely irrelevant to software engineering, only respond with \"Sorry, I can't assist with that.\"\nKeep your answers short and impersonal.\nYou can answer general programming questions and perform the following tasks: \n* Ask a question about the files in your current workspace\n* Explain how the code in your active editor works\n* Make changes to existing code\n* Review the selected code in your active editor\n* Generate unit tests for the selected code\n* Propose a fix for the problems in the selected code\n* Scaffold code for a new file or project in a workspace\n* Create a new Jupyter Notebook\n* Find relevant code to your query\n* Propose a fix for the a test failure\n* Ask questions about VS Code\n* Generate query parameters for workspace search\n* Run a command\n* Ask how to do something in the terminal\n* Explain what just happened in the terminal\n* Propose a fix for the problems in the selected code\n* Explain how the code in your active editor works\n* Review the selected code in your active editor\n* Generate unit tests for the selected code\n* Propose a fix for the a test failure\nYou use the o1-preview version of OpenAI's GPT models.\nFirst think step-by-step - describe your plan for what to build in pseudocode, written out in great detail.\nThen output the code in a single code block.\nMinimize any other prose.\nUse Markdown formatting in your answers.\nMake sure to include the programming language name at the start of the Markdown code blocks.\nAvoid wrapping the whole response in triple backticks.\nThe user works in an IDE called Visual Studio Code which has a concept for editors with open files, integrated unit test support, an output pane that shows the output of running the code as well as an integrated terminal.\nThe user is working on a Mac machine. Please respond with system specific commands if applicable.\nThe active document is the source code the user is looking at right now.\nYou can only give one reply for each conversation turn.\n"},{"role":"user","content":"How to implement simple redundancy and create margin of safety for writing to flash memory for data persistence purpose?"}],"model":"o1-preview","stream":false}

HTTP/2.0 200 
content-security-policy: default-src 'none'; sandbox
content-type: application/json
strict-transport-security: max-age=31536000
x-request-id: 495dff70-2dae-4917-8caf-bf0b5ff647c3
date: Wed, 25 Sep 2024 14:26:26 GMT
x-github-backend: Kubernetes
content-length: 3264

{"choices":[{"content_filter_results":{"hate":{"filtered":false,"severity":"safe"},"self_harm":{"filtered":false,"severity":"safe"},"sexual":{"filtered":false,"severity":"safe"},"violence":{"filtered":false,"severity":"safe"}},"finish_reason":"stop","index":0,"message":{"content":"First, we'll plan the implementation in pseudocode:\n\n1. Define a data structure that includes the data and a checksum for integrity verification.\n2. Implement a function to compute a checksum (e.g., CRC32) of the data.\n3. Create functions to write and read data to and from flash memory with redundancy:\n   - When writing, store multiple copies of the data along with the checksum at different flash memory locations.\n   - When reading, retrieve the data from each location, verify the checksum, and use the valid copy.\n\nNow, here's the code:\n\n```c\n// C code\n\n#include \u003cstdint.h\u003e\n#include \u003cstring.h\u003e\n\n// Function prototypes for flash memory operations (to be implemented according to your hardware)\nint flash_write(uint32_t address, const void *data, size_t length);\nint flash_read(uint32_t address, void *data, size_t length);\n\n// Function to compute a simple checksum (e.g., sum of bytes)\nuint32_t compute_checksum(const uint8_t *data, size_t length) {\n    uint32_t checksum = 0;\n    for (size_t i = 0; i \u003c length; i++) {\n        checksum += data[i];\n    }\n    return checksum;\n}\n\n// Data structure to store data with checksum\ntypedef struct {\n    uint32_t checksum;\n    uint8_t data[256]; // Adjust the size as needed\n} DataBlock;\n\n// Define redundant storage addresses\n#define NUM_COPIES 2\nconst uint32_t FLASH_ADDRESSES[NUM_COPIES] = { 0x08000000, 0x08001000 };\n\n// Function to write data with redundancy\nint write_redundant_data(const uint8_t *data, size_t length) {\n    if (length \u003e sizeof(((DataBlock *)0)-\u003edata)) {\n        return -1; // Data too large\n    }\n\n    DataBlock block;\n    memcpy(block.data, data, length);\n    block.checksum = compute_checksum(block.data, length);\n\n    // Write to multiple flash locations\n    for (int i = 0; i \u003c NUM_COPIES; i++) {\n        if (flash_write(FLASH_ADDRESSES[i], \u0026block, sizeof(DataBlock)) != 0) {\n            return -1; // Write failed\n        }\n    }\n    return 0; // Success\n}\n\n// Function to read data with redundancy\nint read_redundant_data(uint8_t *data, size_t *length) {\n    DataBlock block;\n    for (int i = 0; i \u003c NUM_COPIES; i++) {\n        if (flash_read(FLASH_ADDRESSES[i], \u0026block, sizeof(DataBlock)) == 0) {\n            uint32_t checksum = compute_checksum(block.data, sizeof(block.data));\n            if (checksum == block.checksum) {\n                memcpy(data, block.data, sizeof(block.data));\n                if (length) {\n                    *length = sizeof(block.data);\n                }\n                return 0; // Valid data found\n            }\n        }\n    }\n    return -1; // No valid data found\n}\n```","role":"assistant"}}],"created":1727274369,"id":"chatcmpl-ABNFxWDOLnZ1VimWfgKxlzAVQrCbH","model":"o1-preview-2024-09-12","prompt_filter_results":[{"content_filter_results":{},"prompt_index":0}],"usage":{"completion_tokens":2311,"prompt_tokens":588,"total_tokens":2899}}

Here's the request and response when I'm asking the same question in the nvim plugin

POST https://api.githubcopilot.com/chat/completions HTTP/2.0
accept: */*
accept-encoding: deflate, gzip
openai-intent: conversation-panel
editor-version: Neovim/0.10.1
openai-organization: github-copilot
copilot-integration-id: vscode-chat
content-type: application/json
user-agent: CopilotChat.nvim/2.0.0
editor-plugin-version: CopilotChat.nvim/2.0.0
content-length: 2410

{"messages":[{"role":"system","content":"You are an AI programming assistant.\nWhen asked for your name, you must respond with \"GitHub Copilot\".\nFollow the user's requirements carefully & to the letter.\nFollow Microsoft content policies.\nAvoid content that violates copyrights.\nIf you are asked to generate content that is harmful, hateful, racist, sexist, lewd, violent, or completely irrelevant to software engineering, only respond with \"Sorry, I can't assist with that.\"\nKeep your answers short and impersonal.\nYou can answer general programming questions and perform the following tasks: \n* Ask a question about the files in your current workspace\n* Explain how the code in your active editor works\n* Generate unit tests for the selected code\n* Propose a fix for the problems in the selected code\n* Scaffold code for a new workspace\n* Create a new Jupyter Notebook\n* Find relevant code to your query\n* Propose a fix for the a test failure\n* Ask questions about Neovim\n* Generate query parameters for workspace search\n* Ask how to do something in the terminal\n* Explain what just happened in the terminal\nYou use the GPT-4 version of OpenAI's GPT models.\nFirst think step-by-step - describe your plan for what to build in pseudocode, written out in great detail.\nThen output the code in a single code block. This code block should not contain line numbers (line numbers are not necessary for the code to be understood, they are in format number: at beginning of lines).\nMinimize any other prose.\nUse Markdown formatting in your answers.\nMake sure to include the programming language name at the start of the Markdown code blocks.\nAvoid wrapping the whole response in triple backticks.\nThe user works in an IDE called Neovim which has a concept for editors with open files, integrated unit test support, an output pane that shows the output of running the code as well as an integrated terminal.\nThe user is working on a Darwin machine. Please respond with system specific commands if applicable.\nThe active document is the source code the user is looking at right now.\nYou can only give one reply for each conversation turn.\n"},{"role":"user","content":"How to implement simple redundancy and create margin of safety for writing to flash memory for data persistence purpose?"}],"top_p":1,"stream":true,"intent":true,"temperature":0.1,"model":"o1-preview-2024-09-12","n":1}

HTTP/2.0 400 
content-security-policy: default-src 'none'; sandbox
content-type: text/plain; charset=utf-8
strict-transport-security: max-age=31536000
x-content-type-options: nosniff
date: Wed, 25 Sep 2024 14:30:16 GMT
content-length: 12
x-github-backend: Kubernetes

Bad Request

EDIT:

Here's a few things that I noticed:

• VSCode doesn't enable streamed response ("stream":false).
• The model name is o1-preview in VSCode
• There's no n, top_p, and intent property on the request sent by VSCode.

[pidgeon777]

I was lucky enough to be admitted into their limited preview program

Me also, I vote for o1 models support 👍.

[4DRIAN0RTIZ]

Has anyone managed to get it to work?

[pidgeon777]

I have not yet attempted this, but incorporating support for this feature would be highly beneficial. Those who have a subscription to GitHub Copilot also have requests to the o1 models included. This means it is possible to make requests to those models 'for free,' without paying extra.. This enhancement could elevate Copilot to be on par with Claude 3.5 Sonnet in terms of coding capabilities.

[gptlang]

I just got access today. Will see if I can get it working

[gptlang]

Took almost an hour but o1 pretty much works now. Thank you @alwint3r for the mitm. Didn't even have to install vscode.

[pidgeon777]

Took almost an hour but o1 pretty much works now. Thank you @alwint3r for the mitm. Didn't even have to install vscode.

Can't wait to try it. Thank you!

[alwint3r]

Took almost an hour but o1 pretty much works now. Thank you @alwint3r for the mitm. Didn't even have to install vscode.

Awesome! I'll try the update right away. Thank you! @gptlang