feat: support for multiple providers

[meridionaljet]

Would be awesome to see this merged!

[tombowditch]

Would love to see Anthropic (claude) added - docs are https://docs.anthropic.com/claude/reference/messages_post

URL seems to be https://api.anthropic.com/v1/messages, very similar format to OpenAI

Models: https://docs.anthropic.com/claude/docs/models-overview

[teto]

testing this branch I get on startup:

Gp: response from the config.providers.copilot.secret command { "bash", "-c", "cat ~/.config/github-copilot/hosts.json | sed -e 's/.*oauth_token...//;s/\".*//'" } is empty

because I dont use copilot and this file doesn't exist.

Also maybe I configure it wrong but my agents have disappeared and instead I get the default as completion I suppose:

:GpAgent                                                                                                                                                                                                                                          
ChatGPT4            CodeCopilot         CodeGPT3-5          CodeGPT4            CodeOllamaDeepSeek  

[teto]

answering to myself, my agents got silently discarded because I had removed the model and/or prompt. It would be best to warn the user rather than discard the agent IMO (at minimum have it in checkhealth). Debugging that I noticed agents and providers are merged, but wouldn't it best to override the default and if user wants the default they can use unpack(default_config.agents) ? Mentioning this here before a merge

[Robitx]

@teto Matthieu, thanks for the review. What are you using without model and sys prompt? Could you provide me with the config you got discarded?

because I dont use copilot and this file doesn't exist.

I'll disable all but OpenAI provider before merging this to main and people will have to explicitly enable others.

I'm considering the unpack for agents, but it seems a bit unwieldy for cases when user wants to override just few defaults while keeping others.

I've used array for agents, so that users wouldn't trip over map syntax for keys with special characters ({ plain = {}, ["special-char"] = {} }), regretting the original decision right now. :slightly_smiling_face:

Anyway, the following will have to be improved before merging.

    -- remove invalid agents
    for name, agent in pairs(M.agents) do
        if type(agent) ~= "table" or not agent.model or not agent.system_prompt then
            M.agents[name] = nil
        end
    end

[teto]

I've commented here https://github.com/Robitx/gp.nvim/issues/120 about the agents getting removed not to pollute this PR.

I noticed that endpoints for image and whisper are hardcoded to openai. I have whisper working with localai locally with http://localhost:11111/v1/audio/transcriptions as url. I dont know enough to recommand anything but if gp.nvim could use a local provider for whisper that would be awesome. Mentioning this in case it impacts the API. For instance if current provider is openai-like, the endpoint would become just the domain name. There is so much stuff going on with LLMs that I dont know how reasonable it is to want to support all ^^'

[januz]

@Robitx Just came across your plugin and it looks fantastic. Is Copilot Chat integration still planned to be included?

[mecattaf]

@teto were you able to figure out how to use gp.nvim with ollama for local whisper?

[teto]

@mecattaf not ollama but local-ai. Should work the same though you can read about my experience at https://github.com/Robitx/gp.nvim/issues/122

[teto]

@Robitx can we merge this ? I would like to contribute to gp.nvim a lot more but while I can deal with stacked PRs, this is a pretty fundamental change that makes it more anxious and I really need the feature xD

[mecattaf]

@teto Looks like this project will no longer be maintained. Such a shame, as it is by far the best experience I have had with LLMs and nvim. Since you have already made the PRs described in this issue, would you be comfortable creating a fork of this project that incorporates the modifications? I think I am not the only user who really wants these features. Thanks!

[gonzaloserrano]

+1

[jonboh]

For anyone interested, I've been using this branch with Ollama for some time (couple of months?) and haven't encountered any error, the chat and code agents work well (with ollama), so maybe forking from here migh be good. I had some problems cleaning up the agent list (so that I only have the three I use), I had to override the already existing agents with empty ones to be able to do that. I'm not super agile with Lua, but I do use this plugin a lot, so I'd be happy to help out.

[1orZero]

When we provide a secret in the provider, do we need to set the OpenAI API key?

[Robitx]

@1orZero if you set secret in the openai provider, it populates openai_api_key automatically, so you don't have to

[1orZero]

Thank you for the quick reply! I tried setting two providers in the config without setting the openai_api_key, but this gives me an error during initialization.

[Robitx]

@1orZero could you send me relevant config snippet?

[1orZero]

This is my config

I am using Lazyvim

if vim.g.vscode then
  return {}
end

local api_config = require("api_config.gp-config")

local agents = {
  {
    name = "ChatGPT4o",
    disable = true,
  },
  {
    name = "ChatGPT4",
    disable = true,
  },
  {
    name = "ChatGPT3-5",
    disable = true,
  },
  {
    name = "CodeGPT4o",
    disable = true,
  },
  {
    name = "CodeGPT4",
    disable = true,
  },
  {
    name = "CodeGPT3-5",
    disable = true,
  },
}
if api_config.azure.agents then
  for _, v in ipairs(api_config.azure.agents) do
    table.insert(agents, v)
  end
end
if api_config.onechat.agents then
  for _, v in ipairs(api_config.onechat.agents) do
    table.insert(agents, v)
  end
end

local config = {
  -- openai_api_key = api_config.azure.secret,
  provider = {
    azure = api_config.azure,
    openai = api_config.onechat,
  },
  agents = agents,
}
return {
  "robitx/gp.nvim",
  config = function()
    require("gp").setup(config)
  end,
}

api-config file :

local system_prompt_chat = "You are a general AI assistant.\n\n"
  .. "The user provided the additional info about how they would like you to respond:\n\n"
  .. "- If you're unsure don't guess and say you don't know instead.\n"
  .. "- Ask question if you need clarification to provide better answer.\n"
  .. "- Think deeply and carefully from first principles step by step.\n"
  .. "- Zoom out first to see the big picture and then zoom in to details.\n"
  .. "- Use Socratic method to improve your thinking and coding skills.\n"
  .. "- Don't elide any code from your output if the answer requires coding.\n"
  .. "- Take a deep breath; You've got this!\n"

local system_prompt_code = "You are an AI working as a code editor.\n\n"
  .. "Please AVOID COMMENTARY OUTSIDE OF THE SNIPPET RESPONSE.\n"
  .. "START AND END YOUR ANSWER WITH:\n\n```"

local azure = {
  endpoint = "https://internal-ai-usages-4.openai.azure.com/openai/deployments/gpt-4o/chat/completions?api-version=2024-02-15-preview",
  secret = "xxxxx",
  agents = {
    {
      name = "ChatGPT4o (Azure)",
      chat = true,
      command = true,
      model = { model = "gpt-4o", temperature = 0.3, top_p = 0.5 },
      system_prompt = system_prompt_chat,
      max_tokens = 4096,
      provider = "openai",
    },
  },
}

local onechat = {
  endpoint = "https://chatapi.onechats.top/v1/chat/completions",
  secret = "xxxxx",
  agents = {
    {
      name = "Claude 3.5 Sonnet (OneChat)",
      chat = true,
      command = true,
      model = { model = "claude-3-5-sonnet-20240620", temperature = 0.3, top_p = 0.5 },
      system_prompt = system_prompt_chat,
      max_tokens = 4096,
      provider = "openai",
    },
  },
}

return {
  azure = azure,
  onechat = onechat,
}

[Robitx]

@1orZero I see two things, it seems you're setting provider instead of providers so the conf doesn't merge properly

  provider = {
    azure = api_config.azure,
    openai = api_config.onechat,
  },

and the ChatGPT4o (Azure) agent has provider set to openai instead of azure?

[1orZero]

@1orZero I see two things, it seems you're setting provider instead of providers so the conf doesn't merge properly

  provider = {
    azure = api_config.azure,
    openai = api_config.onechat,
  },

and the ChatGPT4o (Azure) agent has provider set to openai instead of azure?

You're right! Thank you for reviewing my config. It's now working so well.

Appreciate your effort!

[gonzaloserrano]

@Robitx related to this, to enable ChatGPT4o do I need to disable the rest of agents or just the default ChatGPT3-5? I guess it's something that virtually anyone with a paying openai account will do, so maybe worth adding to the README.

[Robitx]

@gonzaloserrano starting with a clean config, where only openai is active, you can switch between agents via :GpAgent +Tab (or :GpNextAgent which is suitable for shortcuts).

Depending on which window is active, either you'll cycle among code agents or chat agents. The last agent used is persisted on disk.

The commands are mentioned in the readme but i guess it could be improved and made more visible https://github.com/Robitx/gp.nvim?tab=readme-ov-file#agent-commands

The other side of this is that people keep asking for options to specify default agents:

• https://github.com/Robitx/gp.nvim/issues/148
• https://github.com/Robitx/gp.nvim/discussions/85

Which I'll probably have to do in some form.