Feat: Add integration with Groq

[naripok]

Hey David!

All good? :)

Do you take PRs?

And is integration with cloud providers out of scope here?

I wanted to test with Llama3 70b, and since I can't run it locally and groq.com provides some fast and (up to now) free endpoints to it, I thought I'd integrate it here. I know that there are other plugins doing the cloud provider thing, but I like the ergonomics of yours better.

Here is a plugin config example for using it with groq:

local use_local = false

local get_api_key = function(path)
  local expanded_path = vim.fn.expand(path)
  local file = io.open(expanded_path, "r")
  if file then
    local content = file:read("*all"):gsub("\n", "")
    file:close()
    return content
  else
    error("Could not open file!")
  end
end

local base_configs = {
  quit_map = "q", -- set keymap for close the response window
  retry_map = "<c-r>", -- set keymap to re-send the current prompt
  -- list_models = '<omitted lua function>', -- Retrieves a list of model names
  display_mode = "split", -- The display mode. Can be "float" or "split".
  show_prompt = false, -- Shows the prompt submitted to Ollama.
  show_model = false, -- Displays which model you are using at the beginning of your chat session.
  no_auto_close = false, -- Never closes the window automatically.
  debug = false -- Prints errors and the command which is run.
}

local groq_configs = {
  model = "llama3-70b-8192",
  -- model = "mixtral-8x7b-32768",
  body = { max_tokens = nil, temperature = 1, top_p = 1, stop = nil },
  command = function(options)
    local api_url = "https://api.groq.com"
    local api_endpoint = api_url .. "/openai/v1/chat/completions"
    local api_key_path = "~/.groq/creds"
    local api_key = get_api_key(api_key_path)
    local curl_cmd = "curl --silent --no-buffer -X POST"
    local header_options = "-H 'Authorization: Bearer " .. api_key .. "'"
    return curl_cmd .. " " .. header_options .. " " .. api_endpoint .. " -d $body"
  end,
  init = nil
}

local ollama_configs = {
  model = "llama3",
  host = "localhost", -- The host running the Ollama service.
  port = "11434", -- The port on which the Ollama service is listening.
  command = function(options)
    return "curl --silent --no-buffer -X POST http://" .. options.host .. ":" .. options.port .. "/api/chat -d $body"
  end,
  -- Function to initialize Ollama
  -- The command for the Ollama service. You can use placeholders $prompt, $model and $body (shellescaped).
  -- This can also be a command string.
  -- The executed command must return a JSON object with { response, context }
  -- (context property is optional).
  init = function(options) pcall(io.popen, "ollama serve > /dev/null 2>&1 &") end,
}

return {
    "naripok/gen.nvim",
    config = function()
      if use_local then
        require('gen').setup(vim.tbl_deep_extend("force", ollama_configs, base_configs))
      else
        require('gen').setup(vim.tbl_deep_extend("force", groq_configs, base_configs))
      end
    end
}

Anyway, if this is out of scope, just let me know and I'll close the PR. Or you can close it yourself. :)

Cheers!

[oldgalileo]

Got this up and running with the OpenAI API quite easily. Big +1 for this.

Also, just as a heads that from what I've seen dropping the temperature and top_p will yield better code results w/ most models (this conceptually makes a fair bit of sense, but YMMK especially with code fine-tuned models 🤷 )

[naripok]

Got this up and running with the OpenAI API quite easily. Big +1 for this.

I was envisioning putting each of this configs into its own user command too, so I could go :ChatOllama, :ChatClaudeOpus, :ChatLammaGroq, as needed.

They all more-less share API interface, and it would be really nice to have all the LLMs at my fingers at will, so I could use the best (and cheaper) one for the task at any given moment.

Also, for fine-tunes. :ChatOllamaLlama3Code, :ChatOllamaLlama3LongContext...

[bkono]

Got this up and running with the OpenAI API quite easily. Big +1 for this.

I was envisioning putting each of this configs into its own user command too, so I could go :ChatOllama, :ChatClaudeOpus, :ChatLammaGroq, as needed.

They all more-less share API interface, and it would be really nice to have all the LLMs at my fingers at will, so I could use the best (and cheaper) one for the task at any given moment.

Also, for fine-tunes. :ChatOllamaLlama3Code, :ChatOllamaLlama3LongContext...

Just wanted to say thank you for this change, and the sample. While there are other plugins that support multiple providers, something about the flow of this one clicks for me.

I took a slightly different path than using different commands, and iterated on the original sample with dynamic building of the commands, based on the model set on the prompt. This approach defaults prompts to Ollama, but lets me create some that use groq or openai, and holds to the standard keymappings and Gen menu. Sample is available at this gist if anyone is looking to do something similar.

[yofriadi]

I think this should be added in README, searching for issue but found gem!