llm.nvim does not attach to the buffer

[rhusiev]

Some time ago llm.nvim stopped completing anything. I went to the source and added some prints to see, why it does not give ghost text. It turned out that the print "not attached" fired:

function M.get_completions(callback)
  if M.client_id == nil then
    vim.print("no client_id")
    return
  end
  if not lsp.buf_is_attached(0, M.client_id) then
    vim.print("not attached")
    return
  end
  ...

I then dug and found the following autocmd in M.setup:

    api.nvim_create_autocmd("BufEnter", {
      group = augroup,
      pattern = config.get().enable_suggestions_on_files,
      callback = function(ev)
        vim.print("notattaching")
        if not lsp.buf_is_attached(ev.buf, client_id) then
          vim.print("attaching")
          lsp.buf_attach_client(ev.buf, client_id)
        end
      end,
    })

Strangely, the these lines were not run (no prints about attaching). If I change the "BufEnter" to "InsertLeave" those prints work and some requests go to the ollama, as my gpu starts being used (though it still does not show the ghost text for some reason - some other problem)

[rhusiev]

I am using neovim v0.10.0, llm.nvim latest (1dcf519 commit), llm-ls 0.5.3, by the way

[rhusiev]

Apparantly, the llm.nvim has some completion (after I manually attach to the buffer), but it is always "\n".

Here is the sent request:

{                                                                                                                                                     
  backend = "ollama",                                                                                                                                          
  contextWindow = 600,                                                                                                                                         
  disableUrlPathCompletion = false,                                                                                                                            
  fim = {                                                                                                                                                      
    enabled = true,                                                                                                                                            
    middle = "<|fim_middle|>",                                                                                                                                 
    prefix = "<|fim_prefix|>",                                                                                                                                 
    suffix = "<|fim_suffix|>"                                                                                                                                  
  },                                                                                                                                                           
  ide = "neovim",                                                                                                                                              
  model = "codegemma:2b-code-q4_K_M",                                                                                                                          
  position = {                                                                                                                                                 
    character = 5,                                                                                                                                             
    line = 11                                                                                                                                                  
  },                                                                                                                                                           
  requestBody = {                                                                                                                                              
    options = {                                                                                                                                                
      num_predict = 4,                                                                                                                                         
      temperature = 0.2,                                                                                                                                       
      top_p = 0.95                                                                                                                                             
    }                                                                                                                                                          
  },                                                                                                                                                           
  textDocument = {                                                                                                                                             
    uri = "file:///.../temp/py.py"                                                                                                                     
  },                                                                                                                                                           
  tlsSkipVerifyInsecure = false,                                                                                                                               
  tokenizerConfig = {                                                                                                                                          
    repository = "google/codegemma-2b"                                                                                                                         
  },                                                                                                                                                           
  tokensToClear = { "<|endoftext|>", "<|file_separator|>" },                                                                                                   
  url = "http://localhost:11434/api/generate"                                                                                                                  
}

And the received:

{                                                                                                                                                    
  completions = { {                                                                                                                                            
      generated_text = "\n"                                                                                                                                    
    } },                                                                                                                                                       
  request_id = "..."                                                                                                          
}

[rhusiev]

It seems that the problem with "\n" had something to do with codegemma, as other models give at least something (which is strange, because codegemma worked previously with the same config). However, there is still the problem of llm.nvim not attaching to the buffer

[rhusiev]

It seems that the tokenizer does not work or something, as codegemma and deepseek coder base do not work (the ones with custom suffixes, prefixes and tokenizers, not mentioned in the readme). whereas refact (which uses the same suffixes and prefixes as starcoder, which is mentioned in readme) works

[rhusiev]

Though it is definitely a separate thing from not attaching to the buffer