search

microsoft / vscode-copilot-release

Feedback on GitHub Copilot Chat UX in Visual Studio Code.
https://marketplace.visualstudio.com/items?itemName=GitHub.copilot-chat
Creative Commons Attribution 4.0 International
322 stars 29 forks source link

Sorry, the response hit the length limit. Please rephrase your prompt #534

Closed ismir2020 closed 3 weeks ago

ismir2020 commented 11 months ago

Steps to Reproduce:

  1. Input Prompt: Please refactor the code of reset.blade.php to match the design of register.blade.php

Answer: output.txt

Im not sure but is it possible to user gpt3.5-16k? And another thing is that the output doesnt reach the 4k limit or the 8k limit length but still getting this error

lyc280705 commented 11 months ago

Prehaps they just want to reduce the expenses. When I was using copilot, it also told me that I had reached the limit and asked me to try again later. This kind of thing never happened before I upgraded the extension to v0.10.0. And copilot is more likely to give pseudocode without a following code now.

isi2010 commented 11 months ago

But this can’t be the issue. There should be an error while creating the answer

lramos15 commented 11 months ago

Do you see this on the first message in the history? How long is register.blade.php?

ismir2020 commented 11 months ago

Do you see this on the first message in the history? How long is register.blade.php?

Yes also in first message but mostly within a longer chat. When deleting the chat history it works again for some time. Then it again not working when deleting chat. Sometimes also works when creating some other code and then return back to the prompt and send the exact prompt again and its working.

register.blade.php has 166 words, 3990 letters or 1246 letters without spaces. So not so long for gpt3.5 or gp4. Same prompt (including full code of register.blade.php) send to the chat.openai was generating complete output with 3.5 and gpt4 without any errors

lramos15 commented 11 months ago

Very odd. I'm not seeing any finish reason of "length" in the logs you posted either. Is the php file sensitive data or would you be willing to share it?

ismir2020 commented 11 months ago

reset.blade.php: `@extends('layouts.app')

@section('content')

{{ __('Reset Password') }}

@csrf
@error('email') {{ $message }} @enderror
@error('password') {{ $message }} @enderror

@endsection`

ismir2020 commented 11 months ago

Now got again the issue. File is 44 lines long/short

2023-11-08T06:41:17.826Z [INFO] [streamMessages] message 0 returned. finish reason: [stop] 2023-11-08T06:41:17.827Z [INFO] [streamChoices] request done: headerRequestId: [032b29c5-b096-4e84-9c26-37f56d8d04bc] model deployment ID: [x2191bfb807ec] 2023-11-08T06:43:43.216Z [INFO] [chat fetch] engine https://api.githubcopilot.com/chat 2023-11-08T06:43:43.216Z [INFO] [chat fetch] modelMaxTokenWindow 4096 2023-11-08T06:43:43.216Z [INFO] [chat fetch] chat model gpt-4 2023-11-08T06:43:46.423Z [INFO] [chat fetch] request.response: [https://api.githubcopilot.com/chat/completions], took 3206 ms 2023-11-08T06:44:08.488Z [INFO] [streamMessages] message 0 returned. finish reason: [length] 2023-11-08T06:44:08.489Z [INFO] [streamChoices] request done: headerRequestId: [575aec11-89c6-474c-afaf-4acb3a53168e] model deployment ID: [x3b0892d9e5fc]

isi2010 commented 10 months ago

Any updates to this ?

alexrosen commented 10 months ago

I had this issue, but I was able to get what I needed after doing clear chat. It would be helpful for the message to suggest that as an option along with the "rephrase" option.

diablodale commented 10 months ago

Today 19 November 2023 I am repeatedly getting this same error Sorry, the response hit the length limit. Please rephrase your prompt. Until today, I don't think I have seen this error but maybe once. It is very very rare....until today. I suspect something server-side or config-side has changed for the worse.

My scenario is to ask copilot to write a needed function in chat. The function it output is only 50 lines. And only a sentence of text before and after that code.

In addition, the VSCode copilot chat UI fades the reply from Copilot from full to almost invisible...making a perfectly usable reply difficult to see and read. Luckily, I can copy/paste the errantly dimmed content.

lramos15 commented 10 months ago

Until today, I don't think I have seen this error but maybe once. It is very very rare....until today. I suspect something server-side or config-side has changed for the worse.

Does clearing your chat help? I haven't been able to hit this even when asking chat to be very verbose so I'm trying to figure out what might be going on here

diablodale commented 10 months ago

@lramos15 Today, I am not seeing the length error. Between days Ive restarted vscode and I saw the copilto extension updated itself. I've been using it today for bth writing code and discussing topics with no length errors.

For future debug, is there a way to get diags like accumulated tokens/length from copilot or submit a trackingId for later internal analysis?

lramos15 commented 10 months ago

For future debug, is there a way to get diags like accumulated tokens/length from copilot or submit a trackingId for later internal analysis?

CMD / CTRL + SHIFT + U -> Copilot Chat and pasting that output helps

diablodale commented 10 months ago

I just got the length error. Here is paste from the output window you requested...

...
2023-11-25T00:38:18.802Z [INFO] [auth] Got Copilot token for diablodale
2023-11-25T01:00:32.123Z [INFO] [chat fetch] engine https://api.githubcopilot.com/chat
2023-11-25T01:00:32.123Z [INFO] [chat fetch] modelMaxTokenWindow 4096
2023-11-25T01:00:32.123Z [INFO] [chat fetch] chat model gpt-4
2023-11-25T01:00:36.608Z [INFO] [chat fetch] request.response: [https://api.githubcopilot.com/chat/completions], took 4484 ms
2023-11-25T01:01:06.221Z [INFO] [streamMessages] message 0 returned. finish reason: [length]
2023-11-25T01:01:06.222Z [INFO] [streamChoices] request done: headerRequestId: [1493a27a-57cb-4a17-a551-e0ebfd847854] model deployment ID: [xfbad2a5f462a]

Again the copilot vscode extention derp dims the reply (why? its valid and good...and now I can't see it). I can barely see that it has a complete intact block of code. And it has an ending sentence though that sentence is truncated.

After seeing this error twice, I typed /clear and then used the up arrow to ask the exact same question. This time it worked. And I also noticed in the copilot window the 4096 changed to 8192 on some lines.

2023-11-25T00:38:18.802Z [INFO] [auth] Got Copilot token for diablodale
2023-11-25T01:00:32.123Z [INFO] [chat fetch] engine https://api.githubcopilot.com/chat
2023-11-25T01:00:32.123Z [INFO] [chat fetch] modelMaxTokenWindow 4096
2023-11-25T01:00:32.123Z [INFO] [chat fetch] chat model gpt-4
2023-11-25T01:00:36.608Z [INFO] [chat fetch] request.response: [https://api.githubcopilot.com/chat/completions], took 4484 ms
2023-11-25T01:01:06.221Z [INFO] [streamMessages] message 0 returned. finish reason: [length]
2023-11-25T01:01:06.222Z [INFO] [streamChoices] request done: headerRequestId: [1493a27a-57cb-4a17-a551-e0ebfd847854] model deployment ID: [xfbad2a5f462a]
2023-11-25T01:03:18.816Z [INFO] [auth] Logged in as diablodale
2023-11-25T01:03:19.218Z [INFO] [chat] copilot token chat_enabled: true
2023-11-25T01:03:19.218Z [INFO] [auth] Got Copilot token for diablodale
2023-11-25T01:07:04.327Z [INFO] [chat fetch] engine https://api.githubcopilot.com/chat
2023-11-25T01:07:04.327Z [INFO] [chat fetch] modelMaxTokenWindow 4096
2023-11-25T01:07:04.327Z [INFO] [chat fetch] chat model gpt-4
2023-11-25T01:07:07.052Z [INFO] [chat fetch] request.response: [https://api.githubcopilot.com/chat/completions], took 2725 ms
2023-11-25T01:07:07.056Z [INFO] [streamMessages] message 0 returned. finish reason: [stop]
2023-11-25T01:07:07.057Z [INFO] [streamChoices] request done: headerRequestId: [959f4daa-9484-449f-9172-c0c4a4848ced] model deployment ID: [xfbad2a5f462a]
2023-11-25T01:07:07.077Z [INFO] [chat fetch] engine https://api.githubcopilot.com/chat
2023-11-25T01:07:07.077Z [INFO] [chat fetch] modelMaxTokenWindow 8192
2023-11-25T01:07:07.077Z [INFO] [chat fetch] chat model gpt-3.5
2023-11-25T01:07:08.281Z [INFO] [chat fetch] request.response: [https://api.githubcopilot.com/chat/completions], took 1203 ms
2023-11-25T01:07:08.282Z [INFO] [streamMessages] message 0 returned. finish reason: [stop]
2023-11-25T01:07:08.283Z [INFO] [streamChoices] request done: headerRequestId: [73d64b15-8250-4261-ab02-b58c904b7a71] model deployment ID: [xba94157a3481]
2023-11-25T01:07:38.766Z [INFO] [chat fetch] engine https://api.githubcopilot.com/chat
2023-11-25T01:07:38.766Z [INFO] [chat fetch] modelMaxTokenWindow 4096
2023-11-25T01:07:38.766Z [INFO] [chat fetch] chat model gpt-4
2023-11-25T01:07:40.640Z [INFO] [chat fetch] request.response: [https://api.githubcopilot.com/chat/completions], took 1873 ms
2023-11-25T01:08:02.860Z [INFO] [streamMessages] message 0 returned. finish reason: [length]
2023-11-25T01:08:02.863Z [INFO] [streamChoices] request done: headerRequestId: [8e5effcd-2052-43a2-b4ef-79e2205289e2] model deployment ID: [xfbad2a5f462a]
2023-11-25T01:08:41.620Z [INFO] [chat fetch] engine https://api.githubcopilot.com/chat
2023-11-25T01:08:41.620Z [INFO] [chat fetch] modelMaxTokenWindow 4096
2023-11-25T01:08:41.620Z [INFO] [chat fetch] chat model gpt-4
2023-11-25T01:08:43.719Z [INFO] [chat fetch] request.response: [https://api.githubcopilot.com/chat/completions], took 2099 ms
2023-11-25T01:08:43.722Z [INFO] [streamMessages] message 0 returned. finish reason: [stop]
2023-11-25T01:08:43.723Z [INFO] [streamChoices] request done: headerRequestId: [1297417a-1319-4e5f-9ed3-df9b6f11ba18] model deployment ID: [xfbad2a5f462a]
2023-11-25T01:08:43.749Z [INFO] [chat fetch] engine https://api.githubcopilot.com/chat
2023-11-25T01:08:43.749Z [INFO] [chat fetch] modelMaxTokenWindow 8192
2023-11-25T01:08:43.749Z [INFO] [chat fetch] chat model gpt-3.5
2023-11-25T01:08:44.743Z [INFO] [chat fetch] request.response: [https://api.githubcopilot.com/chat/completions], took 994 ms
2023-11-25T01:08:44.744Z [INFO] [streamMessages] message 0 returned. finish reason: [stop]
2023-11-25T01:08:44.744Z [INFO] [streamChoices] request done: headerRequestId: [eb256671-11d1-4e4f-8a5d-c3aec07220cc] model deployment ID: [xba94157a3481]
2023-11-25T01:08:58.818Z [INFO] [chat fetch] engine https://api.githubcopilot.com/chat
2023-11-25T01:08:58.818Z [INFO] [chat fetch] modelMaxTokenWindow 4096
2023-11-25T01:08:58.818Z [INFO] [chat fetch] chat model gpt-4
2023-11-25T01:09:01.627Z [INFO] [chat fetch] request.response: [https://api.githubcopilot.com/chat/completions], took 2808 ms
2023-11-25T01:09:20.769Z [INFO] [streamMessages] message 0 returned. finish reason: [length]
2023-11-25T01:09:20.772Z [INFO] [streamChoices] request done: headerRequestId: [df846b58-a576-4f44-a695-2174c988dcce] model deployment ID: [xfbad2a5f462a]
2023-11-25T01:09:36.942Z [INFO] [chat fetch] engine https://api.githubcopilot.com/chat
2023-11-25T01:09:36.942Z [INFO] [chat fetch] modelMaxTokenWindow 4096
2023-11-25T01:09:36.942Z [INFO] [chat fetch] chat model gpt-4
2023-11-25T01:09:38.398Z [INFO] [chat fetch] request.response: [https://api.githubcopilot.com/chat/completions], took 1454 ms
2023-11-25T01:09:56.181Z [INFO] [streamMessages] message 0 returned. finish reason: [stop]
2023-11-25T01:09:56.182Z [INFO] [streamChoices] request done: headerRequestId: [8a3c34f7-0553-440b-9143-276e47131dd7] model deployment ID: [xfbad2a5f462a]
2023-11-25T01:09:56.189Z [INFO] [chat fetch] engine https://api.githubcopilot.com/chat
2023-11-25T01:09:56.189Z [INFO] [chat fetch] modelMaxTokenWindow 8192
2023-11-25T01:09:56.189Z [INFO] [chat fetch] chat model gpt-3.5
2023-11-25T01:09:56.728Z [INFO] [chat fetch] request.response: [https://api.githubcopilot.com/chat/completions], took 539 ms
2023-11-25T01:09:56.729Z [INFO] [streamMessages] message 0 returned. finish reason: [stop]
2023-11-25T01:09:56.729Z [INFO] [streamChoices] request done: headerRequestId: [1e203b86-253e-42c3-8936-fef4c93b03c1] model deployment ID: [xba94157a3481]
ismir2020 commented 10 months ago

When we gonna solve this issue? Now three weeks later still no fix for this length response issue....

Input Prompt: Please update my auth.php controller with the function

Reference Code: 

<?php namespace App\Controllers\Auth;

use CodeIgniter\Controller;

class Auth extends Controller
{
    public function login()
    {
        if($this->request->getMethod() == 'post') {
            $email = $this->request->getPost('email');
            $password = $this->request->getPost('password');

            $user1 = $this->getUser('admin_user', $email);
            $user2 = $this->getUser('users', $email);

            if ($user1 && password_verify($password, $user1->password)) {
                session()->set('isLoggedIn', true);
                return redirect()->to('/dashboard');
            } elseif ($user2 && password_verify($password, $user2->password)) {
                session()->set('isLoggedIn', true);
                return redirect()->to('/dashboard');
            } elseif (!$user1 && !$user2) {
                session()->setFlashdata('error', 'User not found');
                return view('Auth/login');
            } else {
                session()->setFlashdata('error', 'Incorrect password');
                return view('Auth/login');
            }
        }

        echo view('Auth/login');
    }

    private function getUser($table, $email)
    {
        $db = \Config\Database::connect();
        $builder = $db->table($table);
        $builder->where('email', $email);
        $query = $builder->get();

        return $query->getFirstRow();
    }
}

Output: 

<?php namespace App\Controllers\Auth;

use CodeIgniter\Controller;

class Auth extends Controller
{
    public function login()
    {
        if($this->request->getMethod() == 'post') {
            $email = $this->request->getPost('email');
            $password = $this->request->getPost('password');

            $user1 = $this->getUser('admin_user', $email);
            $user2 = $this->getUser('users', $email);

            if ($user1 && password_verify($password, $user1->password)) {
                session()->set('isLoggedIn', true);
                return redirect()->to('/dashboard');
            } elseif ($user2 && password_verify($password, $user2->password)) {
                session()->set('isLoggedIn', true);
                return redirect()->to('/dashboard');
            } elseif (!$user1 && !$user2) {
                session()->setFlashdata('error', 'User not found');
                return view('Auth/login');
            } else {
                session()->setFlashdata('error', 'Incorrect password');
                return view('Auth/login');
            }
        }

        echo view('Auth/login');
    }

    public function register()
    {
        if($this->request->getMethod() == 'post') {
            $email = $this->request->getPost('email');
            $password = $this->request->getPost('password');
            $confirm_password = $this->request->getPost('confirm_password');

            if ($password !== $confirm_password) {
                session()->setFlashdata('error', 'Passwords do not match');
                return view('Auth/register');
            }

            $password_hash = password_hash($password, PASSWORD_DEFAULT);

            $db = \Config\Database::connect();
            $builder = $db->table('users');
            $builder->insert([
                'email' => $email,
                'password' => $password_hash
            ]);

            session()->setFlashdata('success', 'Registration successful');
            return redirect()->to('/login');
        }

        echo view('Auth/register');
    }

    private function getUser($table, $email)
    {
        $db = \Config\Database::connect();
        $builder = $db->table($table);
        $builder->where('email', $email);
        $query = $builder->get();

        return $query->getFirstRow();
    }
}

Console: [INFO] [default] [2023-11-27T17:03:12.136Z] request.response: [https://copilot-proxy.githubusercontent.com/v1/engines/copilot-codex/completions] took 328 ms [INFO] [streamChoices] [2023-11-27T17:03:12.136Z] solution 0 returned. finish reason: [stop] [INFO] [streamChoices] [2023-11-27T17:03:12.137Z] request done: headerRequestId: [bb9f0b9a-15be-49ce-b685-10f8900823f0] model deployment ID: [w9bda9c6b4b91] [INFO] [default] [2023-11-27T17:03:13.417Z] [fetchCompletions] engine https://copilot-proxy.githubusercontent.com/v1/engines/copilot-codex [INFO] [default] [2023-11-27T17:03:13.611Z] request.response: [https://copilot-proxy.githubusercontent.com/v1/engines/copilot-codex/completions] took 194 ms [INFO] [streamChoices] [2023-11-27T17:03:13.613Z] solution 0 returned. finish reason: [stop] [INFO] [streamChoices] [2023-11-27T17:03:13.614Z] request done: headerRequestId: [37f1216c-1b45-4382-9063-89f1c0e795d9] model deployment ID: [w9bda9c6b4b91]

diablodale commented 10 months ago

Here is screenshot for the UI dimming aspect of this bug. The dimming could be considered a separate UI bug that manifests due to the length-limit bug. Notice the code is intact and complete, but the progressive dimming makes the UI unworkable. image

lramos15 commented 10 months ago

The dimming could be considered a separate UI bug that manifests due to the length-limit bug.

The dimming has been removed in the latest pre-release

image
isi2010 commented 10 months ago

@lramos15 any updates to this issue or not for the response length limit ?

lramos15 commented 10 months ago

I've gone ahead and deployed a fix for this in the pre-release that is out today. Please try the latest insiders and pre-release and let me know if you still see issues! Thanks for your patience :)

isi2010 commented 10 months ago

@lramos15 Still facing this issue. The output isnt so long. So where's the issue that it gonna be cut? Possible to make a feature like on openAI chat like "Proceed generating"?

Output code: 

<?= $this->extend('Layout/layout.php') ?>

<?= $this->section('content') ?>

    <!-- Your content goes here -->

    <div class="flex min-h-full flex-col justify-center px-6 py-12 lg:px-8">
        <div class="sm:mx-auto sm:w-full sm:max-w-sm">
            <h2 class="mt-10 text-center text-2xl font-bold leading-9 tracking-tight text-gray-900">Register</h2>

            <?php if(session()->getFlashdata('register_error')): ?>
                <div class="bg-red-500 text-white p-3 rounded mt-4" role="alert">
                    <?= session()->getFlashdata('register_error') ?>
                </div>
            <?php endif; ?>

            <?php echo form_open('register', ['class' => 'space-y-6 mt-10']); ?>
                <div>
                    <label for="email" class="block text-sm font-medium leading-6 text-gray-900">Email:</label>
                    <div class="mt-2">
                        <input type="email" id="email" name="email" class="block w-full rounded-md border-0 py-1.5 text-gray-900 shadow-sm ring-1 ring-inset ring-gray-300 placeholder:text-gray-400 focus:ring-2 focus:ring-inset focus:ring-indigo-600 sm:text-sm sm:leading-6" value="<?= set_value('email') ?>" required>
                    </div>
                </div>

                <div>
                    <label for="password" class="block text-sm font-medium leading-6 text-gray-900">Password:</label>
                    <div class="mt-2">
                        <input type="password" id="password" name="password" class="block w-full rounded-md border-0 py-1.5 text-gray-900 shadow-sm ring-1 ring-inset ring-gray-300 placeholder:text-gray-400 focus:ring-2 focus:ring-inset focus:ring-indigo-600 sm:text-sm sm:leading-6" required>
                    </div>
                </div>

                <div>
                    <label for="confirm_password" class="block text-sm font-medium leading-6 text-gray-900">Confirm Password:</label>
                    <div class="mt-2">
                        <input type="password" id="confirm_password" name="confirm_password" class="block w-full rounded-md border-0 py-1.5 text-gray-900 shadow-sm ring-1 ring-inset ring-gray-300 placeholder:text-gray-400 focus:ring-2 focus:ring-inset focus:ring-indigo-600 sm:text-sm sm:leading-6" required>
                    </div>
                </div>

                <div>
                    <button type="submit" name="submit" class="flex w-full justify-center rounded-md bg-indigo-600 px-3 py-1.5 text-sm font-semibold leading-6 text-white shadow-sm hover:bg-indigo-500 focus-visible:outline focus-visible:outline-2 focus-visible:outline-offset-2 focus-visible:outline-indigo-600">Register</button>
                </div>

                <div class="mt-10 text-center text-sm text-gray-500">
                    Already have an account?

Console Log:

[INFO] [streamChoices] [2023-11-29T07:35:18.964Z] solution 0 returned. finish reason: [stop]
[INFO] [streamChoices] [2023-11-29T07:35:18.965Z] request done: headerRequestId: [e6f52a52-81ae-4283-9070-8ec0a1ae5d1f] model deployment ID: [x878ecfd0739c]
[INFO] [default] [2023-11-29T07:35:19.149Z] [fetchCompletions] engine https://copilot-proxy.githubusercontent.com/v1/engines/copilot-codex
[INFO] [default] [2023-11-29T07:35:19.301Z] request.response: [https://copilot-proxy.githubusercontent.com/v1/engines/copilot-codex/completions] took 151 ms
[INFO] [streamChoices] [2023-11-29T07:35:19.301Z] solution 0 returned. finish reason: [stop]
[INFO] [streamChoices] [2023-11-29T07:35:19.302Z] request done: headerRequestId: [b6d2cabd-705e-43fd-90f6-24125c423fe6] model deployment ID: [x878ecfd0739c]
[INFO] [ghostText] [2023-11-29T07:36:25.307Z] Cancelled after extractPrompt
[INFO] [ghostText] [2023-11-29T07:36:25.407Z] Cancelled after extractPrompt
[INFO] [ghostText] [2023-11-29T07:36:25.670Z] Cancelled after extractPrompt
[INFO] [ghostText] [2023-11-29T08:37:16.147Z] Cancelled after extractPrompt
lramos15 commented 10 months ago

@isi2010 Your logs are from the copilot extension and not the Copilot Chat extension

Have you ensured you're on the latest insiders + pre-release?

isi2010 commented 10 months ago

@lramos15 Sorry, i just saw it. Yes i reinstalled today again and the issue still persists... Cant go back so long for the issue at copilot chat

lramos15 commented 10 months ago

Yes i reinstalled today again and the issue still persists

What version are you on?

isi2010 commented 10 months ago

@lramos15 v0.11.2023112901

lramos15 commented 10 months ago

@lramos15 v0.11.2023112901

Can you include your latest logs with the length limit please. I assume you see the blurring is gone?

isi2010 commented 10 months ago

@lramos15 dont have the actual logs cause i deleted chat several times. The blurring isnt gone if the response is too long. Response is about 60 lines of simple code....

lramos15 commented 10 months ago

@lramos15 dont have the actual logs cause i deleted chat several times. The blurring isnt gone if the response is too long. Response is about 60 lines of simple code....

Please provide a screenshot as to what you're seeing

isi2010 commented 10 months ago

Gonna do when the issue comes up again

isi2010 commented 10 months ago

@lramos15

Console log:

2023-11-29T17:11:09.811Z [INFO] [chat fetch] request.response: [https://api.githubcopilot.com/chat/completions], took 948 ms
2023-11-29T17:11:09.814Z [INFO] [streamMessages] message 0 returned. finish reason: [stop]
2023-11-29T17:11:09.815Z [INFO] [streamChoices] request done: headerRequestId: [5e370378-4d15-438d-805b-313943060b1b] model deployment ID: [xba94157a3481]
2023-11-29T17:14:09.751Z [INFO] [chat fetch] engine https://api.githubcopilot.com/chat
2023-11-29T17:14:09.751Z [INFO] [chat fetch] modelMaxTokenWindow 4096
2023-11-29T17:14:09.751Z [INFO] [chat fetch] chat model gpt-4
2023-11-29T17:14:12.582Z [INFO] [chat fetch] request.response: [https://api.githubcopilot.com/chat/completions], took 2831 ms
2023-11-29T17:14:46.915Z [INFO] [streamMessages] message 0 returned. finish reason: [length]
2023-11-29T17:14:46.915Z [INFO] [streamChoices] request done: headerRequestId: [5ca11ae7-fb07-4869-b308-e29405b9ae0a] model deployment ID: [xfbad2a5f462a]

1 2 3

lyc280705 commented 10 months ago

@lramos15

Console log:

2023-11-29T17:11:09.811Z [INFO] [chat fetch] request.response: [https://api.githubcopilot.com/chat/completions], took 948 ms
2023-11-29T17:11:09.814Z [INFO] [streamMessages] message 0 returned. finish reason: [stop]
2023-11-29T17:11:09.815Z [INFO] [streamChoices] request done: headerRequestId: [5e370378-4d15-438d-805b-313943060b1b] model deployment ID: [xba94157a3481]
2023-11-29T17:14:09.751Z [INFO] [chat fetch] engine https://api.githubcopilot.com/chat
2023-11-29T17:14:09.751Z [INFO] [chat fetch] modelMaxTokenWindow 4096
2023-11-29T17:14:09.751Z [INFO] [chat fetch] chat model gpt-4
2023-11-29T17:14:12.582Z [INFO] [chat fetch] request.response: [https://api.githubcopilot.com/chat/completions], took 2831 ms
2023-11-29T17:14:46.915Z [INFO] [streamMessages] message 0 returned. finish reason: [length]
2023-11-29T17:14:46.915Z [INFO] [streamChoices] request done: headerRequestId: [5ca11ae7-fb07-4869-b308-e29405b9ae0a] model deployment ID: [xfbad2a5f462a]

1 2 3

It's not the latest version of copilot extension.You need Visual Studio Code insider to install the latest extension.

isi2010 commented 10 months ago

@lyc280705 @lramos15 Installed the latest insider version (v0.11.2023113001) and still getting this error:

Console:

2023-11-30 17:49:58.891 [info] [streamMessages] message 0 returned. finish reason: [stop]
2023-11-30 17:49:58.891 [info] [streamChoices] request done: requestId: [d975e237-32bd-4bf5-9acc-10b4b5db1346] responseId: [6502641d-bd80-4311-adde-35cf9fc987e4] model deployment ID: [x338b9c029b38]
2023-11-30 17:51:03.880 [info] [chat fetch] engine https://api.githubcopilot.com/chat
2023-11-30 17:51:03.880 [info] [chat fetch] modelMaxTokenWindow 4096
2023-11-30 17:51:03.880 [info] [chat fetch] chat model gpt-4
2023-11-30 17:51:06.555 [info] [chat fetch] request.response: [https://api.githubcopilot.com/chat/completions], took 2674 ms
2023-11-30 17:51:53.574 [info] [streamMessages] message 0 returned. finish reason: [length]
2023-11-30 17:51:53.574 [info] [streamChoices] request done: requestId: [3ec6360d-b1c6-4c0b-97fb-5c18eefe9fb1] responseId: [bf554ba6-9038-444e-9798-e575d6d2cd08] model deployment ID: [x74a4b0e7ace1]

Output:

<body class="m-0 p-0 h-full box-border" x-data="{ isOpen: null, activeDropdown: null }" x-init="$watch('isOpen', val => localStorage.setItem('isOpen', val))">
  <div id="wrapper" class="flex h-full">
    <div id="sidebar" :class="{ 'hidden': !isOpen, 'w-64': isOpen }" class="absolute inset-y-0 left-0 transition-all duration-200 bg-[#F1F5F9] z-10">
      <!-- Sidebar content -->
      <img src="/images/<?= $logo ?>" alt="Logo" class="mx-auto mt-4 w-1/5">
      <ul class="ml-4 mt-4 space-y-2">
        <!-- Sidebar links -->
        <li x-data="{ isOpen: false }" class="relative">
          <button type="button" class="flex items-center justify-between w-full hover:bg-[#DEE9F9] rounded hover:text-[#3B82F6] p-4" aria-controls="dropdown1" data-collapse-toggle="dropdown1" x-on:click="isOpen = !isOpen">
            <span class="flex items-center">
              <span class="w-6 flex justify-center"><i class="fas fa-tachometer-alt"></i></span>
              <span class="mx-2 flex-grow">Dashboard</span>
            </span>
            <span class="w-6 flex justify-end mr-4" :class="{ 'rotate-180': isOpen }"><i class="fas fa-chevron-down"></i></span>
          </button>
          <!-- Dropdown content -->
          <div x-show="isOpen" id="dropdown1" class="w-full mt-0 bg-white shadow p-2 hidden">
            <a href="#sublink1" class="block py-2 px-4 hover:bg-[#DEE9F9] rounded hover:text-[#3B82F6] pl-11">Sublink 1</a>
            <a href="#sublink2" class="block py-2 px-4 hover:bg-[#DEE9F9] rounded hover:text-[#3B82F6] pl-11">Sublink 2</a>
          </div>
        </li>
        <li x-data="{ isOpen: false }" class="relative">
          <button type="button" class="flex items-center justify-between w-full hover:bg-[#DEE9F9] rounded hover:text-[#3B82F6] p-4" aria-controls="dropdown2" data-collapse-toggle="dropdown2" x-on:click="isOpen = !isOpen">
            <span class="flex items-center">
              <span class="w-6 flex justify-center"><i class="fas fa-calendar-alt"></i></span>
              <span class="mx-2 flex-grow">Reservations</span>
            </span>
            <span class="w-6 flex justify-end mr-4" :class="{ 'rotate-180': isOpen }"><i class="fas fa-chevron-down"></i></span>
          </button>
          <!-- Dropdown content -->
          <div x-show="isOpen" id="dropdown2" class="w-full mt-0 bg-white shadow p-2 hidden">
            <a href="#sublink3" class="block py-2 px-4 hover:bg-[#DEE9F9] rounded hover:text-[#3B82F6] pl-11">Sublink 3</a>
            <a href="#sublink4" class="block py-2 px-4 hover:bg-[#DEE9F9] rounded hover:text-[#3B82F6] pl-11">Sublink 4</a>
          </div>
        </li>
        <li x-data="{ isOpen: false }" class="relative">
          <button type="button" class="flex items-center justify-between w-full hover:bg-[#DEE9F9] rounded hover:text-[#3B82F6] p-4" aria-controls="dropdown3" data-collapse-toggle="dropdown3" x-on:click="isOpen = !isOpen">
            <span class="flex items-center">
              <span class="w-6 flex justify-center"><i class="fas fa-users"></i></span>
              <span class="mx-2 flex-grow">Customers</span>
            </span>
            <span class="w-6 flex justify-end mr-4" :class="{ 'rotate-180': isOpen }"><i class="fas fa-chevron-down"></i></span>
          </button>
          <!-- Dropdown content -->
          <div x-show="isOpen" id="dropdown3" class="w-full mt-0 bg-white shadow p-2 hidden">
            <a href="#sublink5" class="block py-2 px-4 hover:bg-[#DEE9F9] rounded hover:text-[#3B82F6] pl-11">Sublink 5</a>
            <a href="#sublink6" class="block py-2 px-4 hover:bg-[#DEE9F9] rounded hover:text-[#3B82F6] pl-11">Sublink 6</a>
          </div>

11

lramos15 commented 10 months ago

Thanks based off the logs you're hitting a normal length limit and everything is working as expected. The problem here is we're including too much history and not enough room for the response when using a small context window model (in this case GPT 4 4k).

In your case I can tell you that you used 3000 tokens for the request, and 1000 tokens for the reply which maxed out the 4k window.

Based on our data this issue is affecting a small set of our users and we're working to remediate it.

isi2010 commented 10 months ago

Is it possible to choose the used modal ? For example gpt35-16k ? Or gpt4-128k?

lramos15 commented 10 months ago

Is it possible to choose the used modal ? For example gpt35-16k ? Or gpt4-128k?

It is not, we constantly use various evaluations and a multi modal approach to provide the best experience.

There's definitely improvements to be made here and it's top of mind at the moment

azurecloudexample commented 4 months ago

Create a new session and again ask Copilot or type commands, and then it will work fine.

lramos15 commented 3 weeks ago

We have increased the limits here substantially and our data is reflecting that in the number of the length limits being hit by users. Let us know what you think and we hope this improves your Copilot experience.