Support different models for auto-complete and manual triggers

[jkunlin]

It is possible to add support for different models for auto-complete and manual triggers. The GPT-4o-min model is very cost-effective and can be used without concern for auto-completion, while GPT-4, though more expensive, can be reserved for manual triggers in more complex situations.

[milanglacier]

Yes, I plan to add another config settings preset which allows the user define different set of config settings and can switch it at runtime.

After the preset is implemented, then the user can easily configure different models (and configure other options like the request timeout) for auto mode and manual mode.

I am still thinking about how to design the preset though, to best integrate with the current configs without too much change.

[milanglacier]

Here is a temporary workaround you can use for now:

cmp.mapping = {
    ['<A-y>'] = function()
        require('minuet').config.provider_options.openai.model = 'gpt-4o'
        cmp.complete {
            config = {
                sources = cmp.config.sources {
                    { name = 'minuet' },
                },
            },
        }
        vim.defer_fn(function()
            require('minuet').config.provider_options.openai.model = 'gpt-4o-mini'
            -- this should be longer than your debounce setting
        end, 500)
    end,
}

[jkunlin]

Here is a temporary workaround you can use for now:

cmp.mapping = {
['<A-y>'] = function()
require('minuet').config.provider_options.openai.model = 'gpt-4o'
cmp.complete {
config = {
sources = cmp.config.sources {
{ name = 'minuet' },
},
},
}
vim.defer_fn(function()
require('minuet').config.provider_options.openai.model = 'gpt-4o-mini'
-- this should be longer than your debounce setting
end, 500)
end,
}

After using this configuration, manual triggers work, but automatic triggers do not. What should the complete configuration look like?

[milanglacier]

require('minuet').setup {
    provider = 'openai',
}

require('cmp').setup {
    sources = {
        {
            { name = 'minuet' },
            -- and your other sources
        }
    },
    mapping = {
        ["A-y"] = function()
        require('minuet').config.provider_options.openai.model = 'gpt-4o'
        cmp.complete {
            config = {
                sources = cmp.config.sources {
                    { name = 'minuet' },
                },
            },
        }
        vim.defer_fn(function()
            require('minuet').config.provider_options.openai.model = 'gpt-4o-mini'
            -- this should be longer than your debounce setting
        end, 500)
    end,
        -- and your other keymappings
    },
}

Note that the debounce and throttle state are shared between manual and auto completion. So if auto complete has triggered, you have to wait for a while (the default is 1s) so that you can trigger the manual complete.

[jkunlin]

It's still not working for me. The manual completion works fine, but the auto-completion doesn't.

[milanglacier]

It's still not working for me. The manual completion works fine, but the auto-completion doesn't.

can you show me your cmp config?

And are you using any distribution like lazyvim?

[jkunlin]

Yes, I am using lazyvim, and my config for cmp is

  {
    "hrsh7th/nvim-cmp",
    ---@param opts cmp.ConfigSchema
    opts = function(_, opts)
      opts.completion.completeopt = "menu,menuone,noselect, noinsert"
      table.insert(opts.sources, { name = "minuet" })
      local cmp = require("cmp")

      opts.mapping = vim.tbl_extend("force", opts.mapping, {
        ["<Tab>"] = cmp.mapping(function(fallback)
          if cmp.visible() then
            cmp.select_next_item()
          else
            fallback()
          end
        end, { "i", "s" }),
        ["<S-Tab>"] = cmp.mapping(function(fallback)
          if cmp.visible() then
            cmp.select_prev_item()
          else
            fallback()
          end
        end, { "i", "s" }),
        ["<CR>"] = cmp.mapping({
          i = function(fallback)
            if cmp.visible() and cmp.get_active_entry() then
              cmp.confirm({ behavior = cmp.ConfirmBehavior.Replace, select = false })
            else
              fallback()
            end
          end,
          s = cmp.mapping.confirm({ select = true }),
          c = cmp.mapping.confirm({ behavior = cmp.ConfirmBehavior.Replace, select = true }),
        }),
        ["<A-y>"] = require("minuet").make_cmp_map(),
      })
    end,
  },

and minuet-ai:

  {
    "milanglacier/minuet-ai.nvim",
    dependencies = {
      "nvim-lua/plenary.nvim",
      "hrsh7th/nvim-cmp",
    },
    config = function()
      require("minuet").setup({
        -- Your configuration options here
        provider = "openai",
        provider_options = {
          openai = {
            model = "gpt-4o-2024-08-06",
            optional = {
              max_tokens = 512,
            },
          },
        },
      })
    end,
  },

I replace ["<A-y>"] = require("minuet").make_cmp_map() as the recipes, but it not working.

[milanglacier]

This is an issue which I have stated in README:

require('cmp').setup { sources = { { { name = 'minuet' }, -- and your other sources } }, performance = { -- It is recommended to increase the timeout duration due to -- the typically slower response speed of LLMs compared to -- other completion sources. This is not needed when you only -- need manual completion. fetching_timeout = 2000, }, }

You should set the fetching_timeout the same as your request timeout (you can adjust the request timeout to a smaller value if you want to use autocomplete, the default is 3s), otherwise the LLM may return the result but cmp simply ignores it.

Besides:

1. I recommend you set the group_index and priority for the minuet source, otherwise the sorting algorithm may rank the LLM completions to bottom. See this section in README.

2. I see you override the CR key in your config, so I recommend you check this section in README,

[jkunlin]

According to this guide, it seems to be working now. Thank you for your patient response.

[isaksamsten]

I just want to echo this request. I have hacked it now to suggest prose completions in LaTeX files but it would be cool to somehow adjust the provider and or prompts based on filetype (or perhaps even treesitter context)

[milanglacier]

I just want to echo this request.

I am working on the feature branch for this and will be merged soon.

it would be cool to somehow adjust the provider and or prompts based on filetype

prompts are already configurable, just pass your own functions. I will make the few_shots to be able to accept a function too, so that everything can be adjusted dynamically, like change based on filetype.

(or perhaps even treesitter context)

As for using treesitter context, there are several challenges:

1. This approach would require specific queries for each filetype.
2. Combined with 1, Using arbitrary nodes without dedicated queries would offer minimal improvement over the current substring method.
3. Controlling context length becomes difficult when combined with treesitter range.

Given these challenges, I do not plan to implement treesitter-based contextl in the near future.

My development plan would be focusing on virtual text and RAG after this issue completed.

[milanglacier]

but it would be cool to somehow adjust the provider and or prompts based on filetype (or perhaps even treesitter context)

I just went through your config for minuet and I have some suggestion on making the config less hackish. Forgive me for stalking your config :)

require('minuet').setup {
    provider_options = {
        openai = {
            system = {
                prompt = function()
                    if vim.bo.ft == 'tex' then
                        return [[your prompt]]
                    elseif vim.bo.ft == 'mail' then
                        return [[your prompt]]
                    else
                        return require('minuet.config').default_template.prompt
                    end
                end,
                guidelines = function()
                    if vim.bo.ft == 'tex' then
                        return [[your prompt]]
                    elseif vim.bo.ft == 'mail' then
                        return [[your prompt]]
                    else
                        return require('minuet.config').default_template.guidelines
                    end
                end,
            },
            few_shots = function()
                if vim.bo.ft == 'tex' then
                    return {
                        -- your few shots examples
                    }
                else
                    return require('minuet.config').default_few_shots
                end
            end,
        },
    },
}

You do not need to copy paste the field that you do not want to override, the fields that you have changed will be merged with the default fields by tbl_deep_extend.