continuedev / continue

⏩ Continue is the leading open-source AI code assistant. You can connect any models and any context to build custom autocomplete and chat experiences inside VS Code and JetBrains
https://docs.continue.dev/
Apache License 2.0
18.82k stars 1.59k forks source link

Gemini Flash response only 2048 character and not continues code generation. #1738

Open robtail opened 3 months ago

robtail commented 3 months ago

Before submitting your bug report

Relevant environment info

- OS: win 11
- Continue: latest - v0.8.43
- IDE: VSCode
- Model: Gemini 1.5 Flash
- config.json:
  {
  "models": [
    {
      "title": "llama3:8b-instruct",
      "provider": "ollama",
      "model": "llama3:8b-instruct-q6_K",
      "temperature": "0.3"
    },
    {
      "title": "Gemini 1.5 Flash",
      "model": "gemini-1.5-flash-latest",
      "contextLength": 1000000,
      "max_output_tokens": 8192,
      "temperature": 0.5,
      "apiKey": "Mykey",
      "provider": "gemini"
    }
  ],
  "customCommands": [
    {
      "name": "test",
      "prompt": "{{{ input }}}\n\nWrite a comprehensive set of unit tests for the selected code. It should setup, run tests that check for correctness including important edge cases, and teardown. Ensure that the tests are complete and sophisticated. Give the tests just as chat output, don't edit any file.",
      "description": "Write unit tests for highlighted code"
    }
  ],
  "tabAutocompleteModel": {
    "title": "deepseek-coder:6.7b-base",
    "provider": "ollama",
    "model": "deepseek-coder:6.7b-base-q6_K"
  },
  "embeddingsProvider": {
    "provider": "ollama",
    "model": "nomic-embed-text"
  }
}


### Description

Hi,

Using Gemini Flash 1.5 model. 
In chat asking to generate a "long" program code (snake game in python.) 
It starts generate the code but stops generate at somewhere. 
I'm pushing  "Continue generation" button but instead to continue the code it starts generating the code again and stops at the same line again. 

How to increase response length? or how force to continue?

This is the prompt issued to continue code generation:
Continue your response exactly where you left off:

### To reproduce

_No response_

### Log output

_No response_
sestinj commented 3 months ago

@robtail You can use the "contextLength" option in your model in config.json to set a longer contextLength, or "completionOptions.maxTokens" for the output length. We'll be doing a better job of automatically detecting this in upcoming releases

robtail commented 3 months ago

Hi Sestinj,

Thanks for the fast reply! :) contextLength was already set to million, see the original bug report: "contextLength": 1000000, And now i tried completionOptions.maxTokens to
"completionOptions.maxTokens": 8192,

but same issue still, it stop generate output after 2054 character. :( and the continue button not continue but start code generate again. image

any suggestion?