Wrong suggestions/errors when using non-standard/Unicode (`[^a-zA-Z]`) macro names

[Andrew15-5]

Issue https://github.com/latex-lsp/texlab/issues/857 (https://github.com/fannheyward/coc-texlab/issues/249) has been resolved and server now provides suggestions with any macro names, hurray! But now there are some bugs with coc-texlab (I'm pretty sure). Below I paste Neovim config, LaTeX file/document and some tests. Only the 3rd test is passed because macro/command name consists of non-special characters ([a-z]). But of course ideally all 7 must be passed (support of 99% any name).

There are some notes:

• tests were conducted using lualatex command (it tolerates/supports a lot more macro names than pdflatex)
• if more than 1 macro has the same digit ([0-9]) as a 1st character, then the compiler error will occur if one of them is used and won't occur if none of them is used
• adding {} to the 1st argument of \newcommand macro (the correct form) will trigger an extension error for some Unicode macro names (I don't know what are the rules for triggering the error)
• I see that default Neovim syntax highlighting also doesn't support some macro names (probably should open an issue there too)
• [New] tests include prior activation of suggestions (automatically or by pressing <C-Space>), without it tests that require pasting from clipboard wouldn't work (see https://github.com/neoclide/coc.nvim/issues/4568)
• [New] if suggest.asciiMatch (:CocCofig) isn't set to false then tests for which you have to paste non-ASCII text from clipboard wouldn't work

init.vim

```vim call plug#begin($HOME . '/.config/nvim/autoload/plugged') Plug 'neoclide/coc.nvim', {'branch': 'release'} Plug 'joshdick/onedark.vim' Plug 'tpope/vim-commentary' call plug#end() colorscheme onedark " Comment lines noremap :Commentary inoremap :Commentary " Use to trigger completion. if has('nvim') inoremap coc#refresh() else inoremap coc#refresh() endif " coc-texlab let g:tex_flavor = 'latex' " Select auto completion option inoremap \ coc#pum#visible() ? coc#pum#next(1) : \ CheckBackspace() ? "\" : \ coc#refresh() inoremap coc#pum#visible() ? coc#pum#prev(1) : "\" ```

bug.tex

```latex \documentclass{article} % \newcommand{\ö}{} % No error % \newcommand{\öö}{} % Error: Unexpected "}" (texlab 1) % \newcommand{\ööabc}{} % Error: Unexpected "}" (texlab 1) \newcommand\ööabc{} \def\öabc{} \def\123{123} \def\abc{} \begin{document} \end{document} ```

Tests

Step 0: `\`, `` 

Test #	Step 1	Step 2
1🔴	`1`  (macro `\1` is undefined, shown otherwise)	`` 
2🔴	`123`, ``  (macro `\123` is defined, shown otherwise)
3🟢	`abc`, ``  (macro `\abc` is defined, the same is shown)
4🔴	`ö`, ``  (macro `\ö` is undefined, shown otherwise)	`` 
5🔴	`öabc`, ``  (macro `\öabc` is defined, shown otherwise)
6🔴	`öö`  (macro `\ööabc` is defined, but no suggestions are shown unless `` is pressed again)	`abc`, ``  (macro `\ööabc` is defined, shown otherwise)
7🔴	`öö`, ``  \*Uncommented 1 line with the extension error* (macros `\ööabc` and `\öö` are defined, but only **word** and **not a macro** `ööabc` is suggested)

\newcommand{}

Meaningless errors are shown which say that the `}` character is inserted in the unexpected/wrong place, but the syntax is correct. Maybe it's only shown when the macro's name have anything appended after the Unicode character (a hunch from just looking at the places where the errors occur).   Output (from `lualatex bug.tex`):  Source code: ```latex \documentclass{article} \newcommand{\ö}{ö} % No error \newcommand{\öö}{öö} % Error: Unexpected "}" (texlab 1) \newcommand{\ööabc}{ööabc} % Error: Unexpected "}" (texlab 1) \renewcommand\ööabc{ööabc} % No error \def\öabc{öabc} \def\123{123} \def\abc{abc} \def\a0{a0} \def\0a{0a} % \def\1a{1a} % OK if no macro starting with \1 is used \begin{document} \ö, \öö, \ööabc, \öabc, \123, \abc, \a0, \0a \end{document} ```

---

Updated test results (checked means no issues):

• [x] test 1 (fixed)
• [x] test 2 (fixed)
• [x] test 3 (was working)
• [x] test 4 (fixed)
• [x] test 5 (fixed)
• [x] test 6 (fixed)
• [x] test 7 (fixed)
• [x] \newcommand{...} (fixed)
• [x] "ultimate" test (fixed)

---

Update: main conversation/issue is now here: https://github.com/latex-lsp/texlab/issues/864. This issue will be closed with the main issue because the issue is related to both projects.

[fannheyward]

Thank you for your report. Do you have tested with the lasted texlab with that fix patch?

[Andrew15-5]

Yes, of course. Just git clone, cd, cargo build and mv texlab /path/to/bin dir/. Works like a charm.

[Andrew15-5]

Updated the comment:

• added short descriptions of what is wrong with each test
• added description for \newcommand{} problem
• changed incorrect/duplicate description of the 7th test
• added plugin to easily manipulate comments

[fannheyward]

Very detailed reproduce steps, will follow up on this issue.

[pfoerster]

@fannheyward It looks like all of the tests are bugs in the server and not in coc-texlab.

@Andrew15-5 Can you try out the latest changes on the texlab master branch, please? The tests should pass now :)

[Andrew15-5]

But now there are some bugs with coc-texlab (I'm pretty sure)

Yay, I'm glad I put this I'm pretty sure.

@pfoerster, let me "transfer" the issue to https://github.com/latex-lsp/texlab, and then we can talk tests. ;)