continuedev / continue

⏩ Continue is the leading open-source AI code assistant. You can connect any models and any context to build custom autocomplete and chat experiences inside VS Code and JetBrains
https://docs.continue.dev/
Apache License 2.0
19.43k stars 1.69k forks source link

OLLAMA_KEEP_ALIVE not working with continue extension #3033

Open xgdgsc opened 7 hours ago

xgdgsc commented 7 hours ago

Before submitting your bug report

Relevant environment info

- OS:Window on arm
- Continue version:v0.8.57
- IDE version:VSCode 1.95.3
- Model:qwen2.5-coder
- config.json:

{
  "models": [
    {
      "model": "claude-3-5-sonnet-latest",
      "provider": "anthropic",
      "apiKey": "",
      "title": "Claude 3.5 Sonnet"
    },
    {
      "model": "AUTODETECT",
      "title": "Autodetect",
      "provider": "ollama"
    }
  ],
  "tabAutocompleteModel": {
    "title": "Codestral",
    "provider": "mistral",
    "model": "codestral-latest",
    "apiKey": ""
  },
  "customCommands": [
    {
      "name": "test",
      "prompt": "{{{ input }}}\n\nWrite a comprehensive set of unit tests for the selected code. It should setup, run tests that check for correctness including important edge cases, and teardown. Ensure that the tests are complete and sophisticated. Give the tests just as chat output, don't edit any file.",
      "description": "Write unit tests for highlighted code"
    }
  ],
  "contextProviders": [
    {
      "name": "code",
      "params": {}
    },
    {
      "name": "docs",
      "params": {}
    },
    {
      "name": "diff",
      "params": {}
    },
    {
      "name": "terminal",
      "params": {}
    },
    {
      "name": "problems",
      "params": {}
    },
    {
      "name": "folder",
      "params": {}
    },
    {
      "name": "codebase",
      "params": {}
    }
  ],
  "slashCommands": [
    {
      "name": "share",
      "description": "Export the current chat session to markdown"
    },
    {
      "name": "cmd",
      "description": "Generate a shell command"
    },
    {
      "name": "commit",
      "description": "Generate a git commit message"
    }
  ]
}

Description

OLLAMA_KEEP_ALIVE not working with continue extension. See https://github.com/ollama/ollama/issues/5272#issuecomment-2489940273 and following comment.

To reproduce

Start ollama serve with OLLAMA_KEEP_ALIVE=-1 environment variable. Chat inside Conitnue Extension. Ollama release model memory after a while instead of keeping it in memory.

Log output

No response