Setting up llamacpp

[rhsimplex]

Hi @gsuuon thanks for sharing this plugin for neovim! I'm new to the ecosystem, so apologies in advance if this questions is obvious. I'm trying to get your plugin running with a local llamacpp server. The llamacpp server is running fine and I can send requests to it. However, when I try to set it up with llm.nvim with the following configuration:

require('lazy').setup({
  'gsuuon/llm.nvim'
})

require('llm').setup({
  default_prompt = require('llm.providers.llamacpp').default_prompt
})

I get the following error when I start nvim:

Error detected while processing /Users/rhsimplex/.config/nvim/init.lua:
E5113: Error while calling lua chunk: vim/shared.lua:0: stack overflow
stack traceback:
        vim/shared.lua: in function ''
        vim/shared.lua: in function 'validate'
        vim/shared.lua: in function ''
        vim/shared.lua: in function ''
        vim/shared.lua: in function ''
        vim/shared.lua: in function ''
        vim/shared.lua: in function ''
        vim/shared.lua: in function ''
        vim/shared.lua: in function ''
        vim/shared.lua: in function ''
        vim/shared.lua: in function ''
        ...
        vim/shared.lua: in function ''
        vim/shared.lua: in function ''
        vim/shared.lua: in function ''
        vim/shared.lua: in function ''
        vim/shared.lua: in function ''
        vim/shared.lua: in function ''
        vim/shared.lua: in function ''
        vim/shared.lua: in function 'tbl_deep_extend'
        ...nderson/.local/share/nvim/lazy/llm.nvim/lua/llm/init.lua:198: in function 'setup'
        /Users/rhsimplex/.config/nvim/init.lua:18: in main chunk

Many thanks if you have any ideas!

[gsuuon]

Hi @rhsimplex! Try putting the llm.nvim setup call into the config field of the plugin spec, so:

require('lazy').setup({
  {
    'gsuuon/llm.nvim',
    config = function()
      require('llm').setup({
        default_prompt = require('llm.providers.llamacpp').default_prompt
      })
    end
  }
})

But I think with lazy you can also just:

require('lazy').setup({
  {
    'gsuuon/llm.nvim',
    opts = {
      default_prompt = require('llm.providers.llamacpp').default_prompt
    }
  }
})

[rhsimplex]

Hi @gsuuon thanks for taking some time to help me out. UnfortunateIy couldn't get it working =( I realize this has something to do with lazy.nvim, and not llm.nvim, so just posting this here in case someone else hits the same issue. I'll keep looking into it. I certainly don't expect you to dedicate any more time to this!

The first approach appears to generate a circular dependency (I assume?):

Failed to run `config` for llm.nvim

vim/shared.lua:0: stack overflow

# stacktrace:
  - vim/shared.lua:0 _in_ ****
  - vim/shared.lua:0 _in_ ****
  - vim/shared.lua:0 _in_ ****
  - vim/shared.lua:0 _in_ ****
  - vim/shared.lua:0 _in_ ****
  ...
  - vim/shared.lua:0 _in_ ****
  - vim/shared.lua:0 _in_ ****
  - vim/shared.lua:0 _in_ ****
  - vim/shared.lua:0 _in_ **tbl_deep_extend**
  - /llm.nvim/lua/llm/init.lua:198 _in_ **setup**
  - .config/nvim/init.lua:27 _in_ **config**
  - .config/nvim/init.lua:23

And the second method cannot locate llm.providers.llamacpp, I suppose because llm.nvim hasn't yet loaded? Sorry I'm not familiar with Lua, so I'm taking a shot in the dark:

Error detected while processing /Users/rhsimplex/.config/nvim/init.lua:
E5113: Error while calling lua chunk: /Users/rhsimplex/.config/nvim/init.lua:19: module 'llm.providers.llamacpp' not found:
        no field package.preload['llm.providers.llamacpp']
        no file './llm/providers/llamacpp.lua'
        no file '/opt/homebrew/share/luajit-2.1/llm/providers/llamacpp.lua'
        no file '/usr/local/share/lua/5.1/llm/providers/llamacpp.lua'
        no file '/usr/local/share/lua/5.1/llm/providers/llamacpp/init.lua'
        no file '/opt/homebrew/share/lua/5.1/llm/providers/llamacpp.lua'
        no file '/opt/homebrew/share/lua/5.1/llm/providers/llamacpp/init.lua'
        no file './llm/providers/llamacpp.so'
        no file '/usr/local/lib/lua/5.1/llm/providers/llamacpp.so'
        no file '/opt/homebrew/lib/lua/5.1/llm/providers/llamacpp.so'
        no file '/usr/local/lib/lua/5.1/loadall.so'
        no file './llm.so'
        no file '/usr/local/lib/lua/5.1/llm.so'
        no file '/opt/homebrew/lib/lua/5.1/llm.so'
        no file '/usr/local/lib/lua/5.1/loadall.so'
stack traceback:
        [C]: in function 'require'
        /Users/rhsimplex/.config/nvim/init.lua:19: in main chunk

(If I load the llm.nvim without opts, it works fine!)

[gsuuon]

I'd expect the first approach at least to work - I'll take a look, might be a bug.

[gsuuon]

@rhsimplex I've pushed a fix, please give it another try and let me know if you're still seeing the issue. It also looks like the correct way to use opts while requiring from the plugin is to provide a function:

require('lazy').setup({
  {
    'gsuuon/llm.nvim',
    opts = function()
      return {
        default_prompt = require('llm.providers.llamacpp').default_prompt
      }
    end
  }
})

[rhsimplex]

Yes thank you! Works great!

[rhsimplex]

Just in case anyone else finds their way here and just wants to use Llama.cpp for basic text completion with lazy, you can use this config:

require('lazy').setup({
  {
    'gsuuon/llm.nvim',
    opts = function()
      return {
        default_prompt = {
      provider = require('llm.providers.llamacpp'),
      mode = 'buffer',
      builder = function(input)
           return {
             prompt = input,
           }
          end
    }
      }
    end
  }
})