Luasnip + texlab not working properly

[tiagovla]

Problem

After using a lua snippet and saving, if I leave the next insert mode a copy of the document will be pasted.

I noticed it only happens with texlab, it works fine with pyright. Also the issue is related to the formatter, it won't happen if it's disabled. I tried to use efm to format instead, but texlab + efm don't work together #485.

System

• Neovim: NVIM v0.6.0-dev+563-gb74916cfd
• TexLab 3.3.0
• Arch

Minimal config

vim.cmd [[packadd packer.nvim]]

require("packer").startup(function()
    use { "wbthomason/packer.nvim" }
    use { "hrsh7th/cmp-path" }
    use { "rafamadriz/friendly-snippets" }
    use { "L3MON4D3/LuaSnip" }
    use {
        "hrsh7th/nvim-cmp",
        config = function()
            local cmp = require "cmp"
            local mapping = {
                ["<C-Space>"] = cmp.mapping.complete(),
                ["<C-e>"] = cmp.mapping.close(),
                ["<CR>"] = cmp.mapping.confirm {
                    behavior = cmp.ConfirmBehavior.Replace,
                    select = true,
                },
            }
            cmp.setup {
                snippet = {
                    expand = function(args)
                        require("luasnip").lsp_expand(args.body)
                    end,
                },
                mapping = mapping,
                completion = { completeopt = "menu,menuone,noinsert" },
                sources = {
                    { name = "luasnip" },
                    { name = "path" },
                },
            }
        end,
    }
    use {
        "neovim/nvim-lspconfig",
        config = function()
            require("lspconfig").texlab.setup {
                cmd = { "/path/to/texlab" },
                log_level = vim.lsp.protocol.MessageType.Log,
                message_level = vim.lsp.protocol.MessageType.Log,
                settings = {
                    texlab = {
                        diagnosticsDelay = 50,
                        -- latexFormatter = "texlab",
                        build = {
                            executable = "latexmk",
                            args = {
                                "-pdf",
                                "-interaction=nonstopmode",
                                "-pvc",
                                "-synctex=1",
                                "-shell-escape",
                                "%f",
                            },
                        },
                        forwardSearch = {
                            args = { "--synctex-forward", "%l:1:%f", "%p" },
                            executable = "zathura",
                        },
                        chktex = { onOpenAndSave = true, onEdit = false },
                        formatterLineLength = 120,
                    },
                },
            }
        end,
    }
    use { "saadparwaiz1/cmp_luasnip" }
end)

-- it also happens manually using the command
vim.cmd [[autocmd BufWritePre * lua vim.lsp.buf.formatting_sync(nil, 1000)]]

require("luasnip/loaders/from_vscode").lazy_load()

Log

lsp.log

[pfoerster]

Thanks for the detailed report. At the moment, I have two suspects:

1. (Incremental) text synchronization
2. latexindent

To find out the cause I have added an additional log statement, which should clear things up. Can you check out the bug/#503-debugging branch and post the texlab log? You can generate a log file using texlab -vvvv --log-file /tmp/texlab.log. Another question that might help: Does completion work normally when this happens? If completion does not work correctly as well, then the incremental text synchronization may be an issue.

[tiagovla]

Completion works, yeah. In fact I only noticed this behavior after I was editing a ~200 line file and after a while it had 15k lines. I tested latexindent with efm without texlab and it works, so I suppose it's related to how texlab is syncing the text after calling it.

Log

texlab.log

[pfoerster]

Thanks for the log file.

That's a bit odd. If the completion is working correctly, then a text synchronization issue (in which the text shown in the editor and the text calculated inside texlab differ) is quite unlikely.

Looking at the log, the text edit calculated by texlab looks fine and even the immediate textDocument/didChange notification seems right.

This line (211) however seems a bit odd:

DEBUG - < {"params":{"textDocument":{"version":39,"uri":"file:\/\/\/home\/tiagovla\/.dotfiles\/neovim\/.config\/nvim\/test.tex"},"contentChanges":[{"rangeLength":1,"range":{"start":{"line":0,"character":23},"end":{"line":1,"character":0}},"text":"\n\n\\begin{document}\n\n\\documentclass{article}\n\\section{ok} % (fold)\n\\label{sec:ok}\n\n% section section name (end)\n\n\\begin{document}\n\n\\documentclass{article}\n"}]},"jsonrpc":"2.0","method":"textDocument\/didChange"}

It re-inserts a lot of text between the first line and the second line after saving. I am not sure if this is intended but it is an incoming notification so it must be something the client did send. Before that, texlab sends this response to the formatting request (line 196):

DEBUG - > {"jsonrpc":"2.0","id":2,"result":[{"newText":"\\documentclass{article}\n\n\\begin{document}\n\n\\documentclass{article}\n\\section{ok} % (fold)\n\\label{sec:ok}\n\n% section section name (end)\n\n\\begin{document}\n\n\\end{document}\n","range":{"end":{"character":0,"line":13},"start":{"character":0,"line":0}}}]}

The response looks good and the following textDocument/didChange notification mirrors the returned text edit:

DEBUG - < {"params":{"textDocument":{"version":36,"uri":"file:\/\/\/home\/tiagovla\/.dotfiles\/neovim\/.config\/nvim\/test.tex"},"contentChanges":[{"rangeLength":144,"range":{"start":{"line":0,"character":23},"end":{"line":12,"character":14}},"text":"\n\n\\begin{document}\n\n\\documentclass{article}\n\\section{ok} % (fold)\n\\label{sec:ok}\n\n% section section name (end)\n\n\\begin{document}\n\n\\end{document}\n"}]},"jsonrpc":"2.0","method":"textDocument\/didChange"}

Immediately after, the client strips the trailing line break:

DEBUG - < {"params":{"textDocument":{"version":37,"uri":"file:\/\/\/home\/tiagovla\/.dotfiles\/neovim\/.config\/nvim\/test.tex"},"contentChanges":[{"rangeLength":1,"range":{"start":{"line":12,"character":14},"end":{"line":13,"character":0}},"text":""}]},"jsonrpc":"2.0","method":"textDocument\/didChange"}

which yields the same document as before when applied. Then a save happens and the large edit is sent to texlab. I am not sure what causes this notification.

[clason]

Neovim just merged a big rewrite of the incremental sync logic. Can you try again with latest master?

[tiagovla]

Neovim just merged a big rewrite of the incremental sync logic. Can you try again with latest master?

I just tested it and the bug is still there. I coded a dockerfile to reproduce the problem, not sure if this might help. Everything is built from source.

Docker

Dockerfile

FROM ubuntu:20.04 AS builder

ARG BUILD_APT_DEPS="ninja-build gettext cargo libtool libtool-bin autoconf automake cmake g++ pkg-config unzip git binutils"
ARG DEBIAN_FRONTEND=noninteractive

RUN apt update && apt upgrade -y && \
    apt install -y ${BUILD_APT_DEPS} && \
    git clone https://github.com/neovim/neovim.git /tmp/neovim && \
    cd /tmp/neovim && \
    make CMAKE_BUILD_TYPE=Release && \
    make CMAKE_INSTALL_PREFIX=/usr/local install && \
    strip /usr/local/bin/nvim && \
    git clone https://github.com/latex-lsp/texlab.git /tmp/texlab && \
    cd /tmp/texlab && \
    git checkout bug/#503-debugging && \
    cargo build --release

FROM ubuntu:20.04
ARG DEBIAN_FRONTEND=noninteractive
RUN apt update && apt upgrade -y && apt install -y git cpanminus make gcc
RUN git clone https://github.com/cmhughes/latexindent.pl.git /latexindent
RUN cpanm YAML::Tiny && cpanm File::HomeDir && cpanm Unicode::GCString
RUN ln -s /latexindent/latexindent.pl /bin/latexindent
COPY --from=builder /usr/local /usr/local/
COPY --from=builder /tmp/texlab/target/release/texlab /usr/bin/texlab
COPY init.lua /root/.config/nvim/
RUN nvim --headless +'autocmd User PackerComplete sleep 100m | qall' +PackerSync
COPY test.tex /

CMD ["/usr/local/bin/nvim"]

init.lua

vim.lsp.set_log_level "debug"

local fn = vim.fn

local function ensure_installed()
    local install_path = fn.stdpath "data" .. "/site/pack/packer/opt/packer.nvim"
    local install = false

    if fn.empty(fn.glob(install_path)) > 0 then
        print "Installing Packer..."
        fn.delete(vim.fn.stdpath "config" .. "/lua/packer_compiled.lua")
        fn.system { "git", "clone", "https://github.com/wbthomason/packer.nvim", install_path }
        install = true
    end

    vim.cmd "packadd packer.nvim"
    return install
end

ensure_installed()

require("packer").startup(function()
    use { "wbthomason/packer.nvim", opt = true }
    use { "hrsh7th/cmp-path" }
    use { "rafamadriz/friendly-snippets" }
    use {
        "L3MON4D3/LuaSnip",
        config = function()
            require("luasnip/loaders/from_vscode").lazy_load()
        end,
    }
    use {
        "hrsh7th/nvim-cmp",
        config = function()
            local cmp = require "cmp"
            local mapping = {
                ["<C-Space>"] = cmp.mapping.complete(),
                ["<C-e>"] = cmp.mapping.close(),
                ["<CR>"] = cmp.mapping.confirm {
                    behavior = cmp.ConfirmBehavior.Replace,
                    select = true,
                },
            }
            cmp.setup {
                snippet = {
                    expand = function(args)
                        require("luasnip").lsp_expand(args.body)
                    end,
                },
                mapping = mapping,
                completion = { completeopt = "menu,menuone,noinsert" },
                sources = {
                    { name = "luasnip" },
                    { name = "path" },
                },
            }
        end,
    }
    use {
        "neovim/nvim-lspconfig",
        config = function()
            require("lspconfig").texlab.setup {
                cmd = { "texlab", "-vvvv", "--log-file", "/tmp/texlab.log" },
                log_level = vim.lsp.protocol.MessageType.Log,
                message_level = vim.lsp.protocol.MessageType.Log,
                settings = {
                    texlab = {
                        diagnosticsDelay = 50,
                        build = {
                            executable = "latexmk",
                            args = {
                                "-pdf",
                                "-interaction=nonstopmode",
                                "-pvc",
                                "-synctex=1",
                                "-shell-escape",
                                "%f",
                            },
                        },
                        forwardSearch = {
                            args = { "--synctex-forward", "%l:1:%f", "%p" },
                            executable = "zathura",
                        },
                        chktex = { onOpenAndSave = false, onEdit = false },
                        formatterLineLength = 120,
                    },
                },
            }
        end,
    }
    use { "saadparwaiz1/cmp_luasnip" }
end)

vim.cmd [[autocmd BufWritePre * lua vim.lsp.buf.formatting_sync(nil, 1000)]]

test.tex

\documentclass{article}

\begin{document}

\end{document}

Instructions

docker build -t bug .
docker run --rm -it bug
:e test.tex

[clason]

Neovim just merged a big rewrite of the incremental sync logic. Can you try again with latest master?

I just tested it and the bug is still there. I coded a dockerfile to reproduce the problem, not sure if this might help. Everything is built from source.

Yes, there still are issues with Neovim's incremental sync logic (which I never hit with texlab+luasnip but you might have) that are actively being worked on atm. Can you try adding the flag allow_incremental_sync=false to your texlab setup and see if that helps?

[tiagovla]

I did try to set allow_incremental_sync=false yesterday and it still happens.

[mjlbach]

I am almost certain vim.cmd [[autocmd BufWritePre * lua vim.lsp.buf.formatting_sync(nil, 1000)]] is the issue. Try removing this. Also this seems like a neovim plugin issue rather than a texlab issue and should be closed IMO.

[tiagovla]

I am almost certain vim.cmd [[autocmd BufWritePre * lua vim.lsp.buf.formatting_sync(nil, 1000)]] is the issue. Try removing this. Also this seems like a neovim plugin issue rather than a texlab issue and should be closed IMO.

@mjlbach How exactly should I format it instead? Removing that autocmd and manually using :lua vim.lsp.buf.formatting_sync() gives me the same behavior.

I also opened the issue here because it doesn't happen with efm, and it seems to be related to latexindent.

[mjlbach]

We need to minimally reproduce what is happening. Luasnip should have nothing to do with our formatting request. If you use a minimal config (such as the one from lspconfig), and directly call vim.lsp.buf.formatting_seq_sync() with only texlab attached to the buffer, do you see incorrect behavior?

[tiagovla]

Obviously it does not happen.

[mjlbach]

Ok, so we've confirmed there is nothing wrong with the built-in client or texlab no?

https://github.com/mjlbach/user_questions/tree/main/texlab_formatting

Doesn't show an issue with me with luasnip installed when just editing/formatting normally, so I'm assuming you have to trigger a snippet insertion, which likely primes a buf_set_text event to fire which overwrites part of your buffer after the snippet is confirmed.

[tiagovla]

Yes, it only happens after triggering a snippet insertion. It does not happen using efm to format it instead of texlab, that's why I assume it's related to how texlab is syncing the buffer after formatting it.

If there was a way to avoid the error after sending textDocument/build to efm so I could disable texlab formatting, it would work just fine for me.

[mjlbach]

Let's completely remove EFM from the equation, that's not important.

Texlab does not sync the buffer, unless it's directly writing to disk (which I think it does not). We (neovim) synchronize the buffer. If you use formatting_sync we should block on formatting. I'll look at this later but I can't easily reproduce your issue.

[tiagovla]

Texlab does not sync the buffer, unless it's directly writing to disk (which I think it does not). We (neovim) synchronize the buffer. If you use formatting_sync we should block on formatting. I'll look at this later but I can't easily reproduce your issue.

Not even with the docker files above?

[pfoerster]

Texlab does not sync the buffer, unless it's directly writing to disk (which I think it does not)

That's correct. texlab never changes a document on the disk directly. In the case of latexindent, we do the formatting with a temporary file and not with the actual file. In the original log, one can see that texlab sends the correct TextEdit (it replaces the whole document with the result of latexindent) as a response to the formatting request.

Initially, texlab receives the correct textDocument/didChange notification. The error occurs after saving with the last textDocument/didChange notification, which duplicates the entire text but texlab is not in control of these messages. These messages are sent from the client to the server and not the other way around. There are two ways that texlab can use to modify the buffer:

1. Responses to requests from the client (like textDocument/formatting)
2. workspace/applyEdit (texlab does not send this request)

Instead, texlab listens to the textDocument/didChange notifications to account for buffer changes. The fact that completion still works, implies that the internal data maintained by texlab is consistent with the buffer on the client side.

[DEBUG][2021-11-07 20:09:26] .../lua/vim/lsp.lua:919    "LSP[texlab]"   "client.request"    1   "textDocument/formatting"   {  options = {    insertSpaces = false,    tabSize = 8  },  textDocument = {    uri = "file:///home/tiagovla/.dotfiles/neovim/.config/nvim/test.tex"  }}    <function 1>    1
[DEBUG][2021-11-07 20:09:26] .../vim/lsp/rpc.lua:339    "rpc.send"  {  id = 2,  jsonrpc = "2.0",  method = "textDocument/formatting",  params = {    options = {      insertSpaces = false,      tabSize = 8    },    textDocument = {      uri = "file:///home/tiagovla/.dotfiles/neovim/.config/nvim/test.tex"    }  }}
[DEBUG][2021-11-07 20:09:26] .../vim/lsp/rpc.lua:446    "rpc.receive"   {  id = 1,  jsonrpc = "2.0",  method = "workspace/configuration",  params = {    items = { {        section = "texlab"      } }  }}
[DEBUG][2021-11-07 20:09:26] .../vim/lsp/rpc.lua:456    "server_request: callback result"   {  result = { {      auxDirectory = ".",      bibtexFormatter = "texlab",      build = {        args = { "-pdf", "-interaction=nonstopmode", "-pvc", "-synctex=1", "-shell-escape", "%f" },        executable = "latexmk",        forwardSearchAfter = false,        onSave = false      },      chktex = {        onEdit = false,        onOpenAndSave = true      },      diagnosticsDelay = 50,      formatterLineLength = 120,      forwardSearch = {        args = { "--synctex-forward", "%l:1:%f", "%p" },        executable = "zathura"      },      latexFormatter = "latexindent",      latexindent = {        modifyLineBreaks = false      }    } },  status = true}
[DEBUG][2021-11-07 20:09:26] .../vim/lsp/rpc.lua:339    "rpc.send"  {  id = 1,  jsonrpc = "2.0",  result = { {      auxDirectory = ".",      bibtexFormatter = "texlab",      build = {        args = { "-pdf", "-interaction=nonstopmode", "-pvc", "-synctex=1", "-shell-escape", "%f" },        executable = "latexmk",        forwardSearchAfter = false,        onSave = false      },      chktex = {        onEdit = false,        onOpenAndSave = true      },      diagnosticsDelay = 50,      formatterLineLength = 120,      forwardSearch = {        args = { "--synctex-forward", "%l:1:%f", "%p" },        executable = "zathura"      },      latexFormatter = "latexindent",      latexindent = {        modifyLineBreaks = false      }    } }}
[DEBUG][2021-11-07 20:09:26] .../vim/lsp/rpc.lua:446    "rpc.receive"   {  id = 2,  jsonrpc = "2.0",  result = { {      newText = "\\documentclass{article}\n\n\\begin{document}\n\\section{ok} % (fold)\n\\label{sec:ok}\n\n% section section name (end)\n\\end{document}\n",      range = {        end = {          character = 0,          line = 8        },        start = {          character = 0,          line = 0        }      }    } }}
[DEBUG][2021-11-07 20:09:26] .../lua/vim/lsp.lua:1074   "on_lines bufnr: 1, changedtick: 35, firstline: 0, lastline: 8, new_lastline: 9, old_byte_size: 124, old_utf32_size: 124, old_utf16_size: 124"  { "\\documentclass{article}", "", "\\begin{document}", "\\section{ok} % (fold)", "\\label{sec:ok}", "", "% section section name (end)", "\\end{document}", "" }
[DEBUG][2021-11-07 20:09:26] .../vim/lsp/rpc.lua:339    "rpc.send"  {  jsonrpc = "2.0",  method = "textDocument/didChange",  params = {    contentChanges = { {        range = {          end = {            character = 14,            line = 7          },          start = {            character = 23,            line = 0          }        },        rangeLength = 100,        text = "\n\n\\begin{document}\n\\section{ok} % (fold)\n\\label{sec:ok}\n\n% section section name (end)\n\\end{document}\n"      } },    textDocument = {      uri = "file:///home/tiagovla/.dotfiles/neovim/.config/nvim/test.tex",      version = 35    }  }}
[DEBUG][2021-11-07 20:09:26] .../lua/vim/lsp.lua:1074   "on_lines bufnr: 1, changedtick: 36, firstline: 8, lastline: 9, new_lastline: 8, old_byte_size: 1, old_utf32_size: 1, old_utf16_size: 1"    {}
[DEBUG][2021-11-07 20:09:26] .../vim/lsp/rpc.lua:339    "rpc.send"  {  jsonrpc = "2.0",  method = "textDocument/didChange",  params = {    contentChanges = { {        range = {          end = {            character = 0,            line = 8          },          start = {            character = 14,            line = 7          }        },        rangeLength = 1,        text = ""      } },    textDocument = {      uri = "file:///home/tiagovla/.dotfiles/neovim/.config/nvim/test.tex",      version = 36    }  }}
[DEBUG][2021-11-07 20:09:26] .../vim/lsp/rpc.lua:339    "rpc.send"  {  jsonrpc = "2.0",  method = "textDocument/didSave",  params = {    textDocument = {      uri = "file:///home/tiagovla/.dotfiles/neovim/.config/nvim/test.tex"    }  }}
[DEBUG][2021-11-07 20:09:26] .../vim/lsp/rpc.lua:446    "rpc.receive"   {  jsonrpc = "2.0",  method = "textDocument/publishDiagnostics",  params = {    diagnostics = {},    uri = "file:///home/tiagovla/.dotfiles/neovim/.config/nvim/test.aux"  }}
[DEBUG][2021-11-07 20:09:26] .../vim/lsp/rpc.lua:446    "rpc.receive"   {  jsonrpc = "2.0",  method = "textDocument/publishDiagnostics",  params = {    diagnostics = {},    uri = "file:///home/tiagovla/.dotfiles/neovim/.config/nvim/test.log"  }}
[DEBUG][2021-11-07 20:09:26] .../vim/lsp/rpc.lua:446    "rpc.receive"   {  jsonrpc = "2.0",  method = "textDocument/publishDiagnostics",  params = {    diagnostics = {},    uri = "file:///home/tiagovla/.dotfiles/neovim/.config/nvim/test.tex"  }}
[DEBUG][2021-11-07 20:09:26] .../vim/lsp/rpc.lua:446    "rpc.receive"   {  jsonrpc = "2.0",  method = "textDocument/publishDiagnostics",  params = {    diagnostics = {},    uri = "file:///home/tiagovla/.dotfiles/neovim/.config/nvim/test.aux"  }}
[DEBUG][2021-11-07 20:09:26] .../vim/lsp/rpc.lua:446    "rpc.receive"   {  jsonrpc = "2.0",  method = "textDocument/publishDiagnostics",  params = {    diagnostics = {},    uri = "file:///home/tiagovla/.dotfiles/neovim/.config/nvim/test.log"  }}
[DEBUG][2021-11-07 20:09:26] .../vim/lsp/rpc.lua:446    "rpc.receive"   {  jsonrpc = "2.0",  method = "textDocument/publishDiagnostics",  params = {    diagnostics = {},    uri = "file:///home/tiagovla/.dotfiles/neovim/.config/nvim/test.tex"  }}
[DEBUG][2021-11-07 20:09:26] .../vim/lsp/rpc.lua:446    "rpc.receive"   {  jsonrpc = "2.0",  method = "textDocument/publishDiagnostics",  params = {    diagnostics = {},    uri = "file:///home/tiagovla/.dotfiles/neovim/.config/nvim/test.aux"  }}
[DEBUG][2021-11-07 20:09:26] .../vim/lsp/rpc.lua:446    "rpc.receive"   {  jsonrpc = "2.0",  method = "textDocument/publishDiagnostics",  params = {    diagnostics = {},    uri = "file:///home/tiagovla/.dotfiles/neovim/.config/nvim/test.log"  }}
[DEBUG][2021-11-07 20:09:26] .../vim/lsp/rpc.lua:446    "rpc.receive"   {  jsonrpc = "2.0",  method = "textDocument/publishDiagnostics",  params = {    diagnostics = { {        code = "24",        message = "Delete this space to maintain correct pagereferences.",        range = {          end = {            character = 1,            line = 4          },          start = {            character = 0,            line = 4          }        },        severity = 2,        source = "chktex"      } },    uri = "file:///home/tiagovla/.dotfiles/neovim/.config/nvim/test.tex"  }}
[DEBUG][2021-11-07 20:09:28] .../lua/vim/lsp.lua:1074   "on_lines bufnr: 1, changedtick: 38, firstline: 0, lastline: 1, new_lastline: 8, old_byte_size: 24, old_utf32_size: 24, old_utf16_size: 24" { "\\documentclass{article}", "", "\\begin{document}", "\\section{ok} % (fold)", "\\label{sec:ok}", "", "% section section name (end)", "\\documentclass{article}" }
[DEBUG][2021-11-07 20:09:28] .../vim/lsp/rpc.lua:339    "rpc.send"  {  jsonrpc = "2.0",  method = "textDocument/didChange",  params = {    contentChanges = { {        range = {          end = {            character = 0,            line = 1          },          start = {            character = 23,            line = 0          }        },        rangeLength = 1,        text = "\n\n\\begin{document}\n\\section{ok} % (fold)\n\\label{sec:ok}\n\n% section section name (end)\n\\documentclass{article}\n"      } },    textDocument = {      uri = "file:///home/tiagovla/.dotfiles/neovim/.config/nvim/test.tex",      version = 38    }  }}

[mjlbach]

@pfoerster I'm sorry, to be clear I don't think this is a texlab issue and believe it's more an issue of a snippet plugin and the returned formatting request fighting for control of the buffer clientside :)

Also to clarify I'm a neovim maintainer/the person (currently) responsible for our built-in language server client.

[pfoerster]

@mjlbach No worries. If there is anything I can do to help troubleshoot the issue from the side of texlab, please let me know.

[tiagovla]

Since it seems to be a client/plugin issue and not a texlab issue. I'm closing it.

[mjlbach]

@tiagovla I wouldn't normally allow this since it's a plugin issue, but you can file it on lspconfig (sans EFM, I know what that issue is) and I'll continue to look into it