support ollama generate reponose when code completions.

carlrobertoh / CodeGPT

JetBrains extension providing access to state-of-the-art LLMs, such as GPT-4, Claude 3, Code Llama, and others, all for free

https://codegpt.ee

Apache License 2.0

912 stars 186 forks source link

support ollama generate reponose when code completions. #517

Open linpan opened 2 months ago

linpan commented 2 months ago

What happened?

curl http://localhost:11434/api/generate -d '{ "model": "llama2", "prompt": "Why is the sky blue?" }' Response A stream of JSON objects is returned:

{ "model": "llama2", "created_at": "2023-08-04T08:52:19.385406455-07:00", "response": "The", "done": false }

error: CleanShot 2024-04-27 at 14 35 23

Relevant log output or stack trace

No response

Steps to reproduce

No response

CodeGPT version

2.4.0

Operating System

None

PhilKes commented 2 months ago

https://github.com/carlrobertoh/CodeGPT/pull/510 will introduce Ollama as a high-level service and use its native API to support code-completions

boswelja commented 2 months ago

To add to that, specifying api/generate doesn't work because the request and response formats don't match OpenAI services. Currently, using OpenAI-compatible APIs is a requirement