David-Kunz / gen.nvim

Neovim plugin to generate text using LLMs with customizable prompts
The Unlicense
977 stars 62 forks source link

add option to preprocess the prompt payload #74

Open albohlabs opened 4 months ago

albohlabs commented 4 months ago

Hi, I could not get my Neovim to talk to my codellama model because the expected format is not the same as the one the plugin produces.

curl -X POST http://localhost:11434/api/generate -d '{
  "model": "codellama",
  "prompt": "Write me a function that outputs the fibonacci sequence"
}'

For this reason I added the preprocess_body option, so that the user of the plugin can easily manipulate the format of the curl post body.

Aaaaaand there is a second commit with some code style changes produced by the lua lsp. Sorry about that. :angel:

Mte90 commented 4 months ago

For codellama it is requested something with this plugin at the end (apart that PR)?

albohlabs commented 4 months ago

Sorry, I don't think I understand what you mean.

But I saw in your dots that you're also using codellama. What is the curl post body that gen.nvim produces? I'm confused. :thinking:

Mte90 commented 4 months ago

I was just trying to understand if with codellama once that this PR is approved it is required something else to add in the plugin settings.

albohlabs commented 3 months ago

The preprocessor function has a default implementation that is used and contains the current prompt creation logic. You can find it at https://github.com/albohlabs/gen.nvim/blob/feat/preprocess-body/lua/gen/init.lua#L35

local default_options = {
    ...
    --- Create the curl prompt that will be included in the substutute of $body
    ---
    ---@param prompt string user input and optionally the highlighted code
    ---@return table
    preprocess_body = function(prompt)
        local messages = {}
        table.insert(messages, { role = "user", content = prompt })
        return { messages = messages }
    end,
   ...

This means that when this PR is merged, there is nothing for you to do. Everything should work out of the box.