Citation completion fails after plugin update

[jakewvincent]

Issue

After recently updating vimtex, citation completion no longer works. Below, I'm providing the output of :PlugDiff after running :PlugUpdate. When I revert these updates, citation completion works fine.

:PlugDiff

- vimtex:
  *   7552bb5 Merge pull request #1750 from slonik-az/fix-1742 (30 hours ago)
  |\
  | * 2c37e88 [issue: #1742] Fixed LaTeX errors in 'test/tests/test-compiler-selected/*.tex' files (31 hours ago)
  |/
  * 7266681 feat: improved custom compile template (2 days ago)
  * 410dcf0 feat: avoid spell checking in \gls macros (2 days ago)
  * 91e3077 chore: improve code (2 days ago)
  * eec393e feat: nospell for urls and acronyms in comments (2 days ago)
  *   a104a3b merge: add compiler vlty (Yalafi + LanguageTool) (2 days ago)
  |\
  | * 264203b Added file name to errorformat (2 days ago)
  | * f7189b2 Added remark for test of vlty installation (2 days ago)
  | * 07ab320 Changed installation error messages (2 days ago)
  | * 9b0509d Direct error message to user on failed installation test (2 days ago)
  | * 3f1b358 Corrected typos (2 days ago)
  | * b7b609a chore: apply suggestions after review (2 days ago)
  | * 7f72218 Default server: 'no';  changed tag *compiler-vlty* (2 days ago)
  | * 5adc529 Changed default server entry to 'no'; fixed bug (2 days ago)
  | * 933fbc7 Added documentation for compiler vlty (2 days ago)
  | * 33462c7 Added vimtex compiler vlty (2 days ago)
  |/
  * c4b33f1 doc: updated docs on VimtexCompileSelected (3 days ago)
  * 455d5de feat: allow template for selection_to_texfile (3 days ago)
  * f5f0cb0 test: add tests for selection_to_texfile (3 days ago)
  * 659e36d chore: improved selection_to_texfile (3 days ago)
  * dfdcdc4 test: increase fudge time for compiler test (5 days ago)
  * 498adaf perf: add caching to find_bibs_blg (5 days ago)
  * 3f92646 test: minor update (5 days ago)
  * b7fbff1 feat: use blg to parse included bib files when possible (5 days ago)
  * cc35896 test: make test files compilable (5 days ago)
  * a09ca70 feat: initialize vimtex in bib files (8 days ago)
  * 8e98c63 test: add tests for bib file state init (8 days ago)
  * 520ecac fix: don't call kpsewhich unless necessary (8 days ago)
  * de135b7 chore: improve state initialization code (8 days ago)
  * 1646940 test: move TestMain to vimtex#test#main (8 days ago)
  * 18eae99 chore: minor refactoring (9 days ago)
  *   b4b5298 merge: add label titles in ToC (9 days ago)
  |\
  | * afb1c25 feat: add label numbering in ToC (9 days ago)
  | * c149aa9 test: add test for label titles in toc (9 days ago)
  | * 10291fc chore: refactor out aux label parser (9 days ago)
  | * d441995 fix: minor bad in toc syntax (9 days ago)
  | * b72a4d3 fix: don't link to Normal syntax group (9 days ago)
  | * c0ac094 chore: update default toc width from 30 -> 50 (9 days ago)
  |/
  * 4437be5 fix: improve ampersand indentation context parser (10 days ago)
  * 9261c7c test: add indentation test with nested envs (10 days ago)
  *   397ecf0 merge: stars in cite commands (10 days ago)
  |\
  | * 5aa776c fix: completion (omni, ncm2, deoplete, and ycm) for \cite*[]{}-like cmds (10 days ago)
  |/
  * 0019dc6 fix: completion pattern for starred cite cmds (2 weeks ago)
  * 773615c test: add test for starred cite comands (2 weeks ago)
  *   49bc335 merge: improve subfile handling (5 weeks ago)
  |\
  | * 9adcb1c doc: updated docs on subfiles package (5 weeks ago)
  | * 5fa29da fix: handle subfiles correctly within ToC (5 weeks ago)
  | * bb2e0e8 fix: handle subfiles import correctly (5 weeks ago)
  | * 3120047 test: add test for ToC with nested subfiles (5 weeks ago)
  | * 6f0afc9 test: add test for VimtexToggleMain in subfiles (5 weeks ago)
  |/
  * 0165b27 test: added test files for #1406 (6 weeks ago)
  * ea959e9 fix: error in XDG_CONFIG_HOME parsing (6 weeks ago)
  * dd1ba42 chore: default XDG_CONFIG_HOME to ~/.config (6 weeks ago)
  * bdc8def fix: silly me (6 weeks ago)
  * 3ac3211 doc: improve docs for coc-vimtex (6 weeks ago)
  *   7ef9c50 merge: simplify/improve gh actions setup and Dockerfile (6 weeks ago)
  |\
  | * 5973701 Simplify installation command in GitHub Actions to use a single `apt install`. (6 weeks ago)
  | * 5045309 Install bibtex and bibparse from Ubuntu packages rather than CPAN (6 weeks ago)
  |/
  * d57d7cf chore: minor simplification (6 weeks ago)
  *   5b537da merge: add option to override existing mappings on conflict (6 weeks ago)
  |\
  | * d577320 Allow vimtex to override existing mappings that use the same key sequences. (6 weeks ago)
  |/
  * 369cc15 test: run Github Actions on PRs (7 weeks ago)
  * 250fe4c fix: allow \lx to properly reload syntax (7 weeks ago)
  * ec5da0d revert: 10c84d0f - does not work as intended (7 weeks ago)
  * afb42ba fix: textobj 'am' should be linewise (7 weeks ago)
  * 10c84d0 Textobj variant targets: map ie,ae also explicitly (7 weeks ago)
  * 34f332a feat: disable indent in markdown envs (7 weeks ago)
  * 666191b feat: g:vimtex_cache_root respects XDG_CACHE_HOME (8 weeks ago)
  * d777cb1 fix: move autoref syntax from cleveref to hyperref (8 weeks ago)
  * 6fbcb8c test: separate hyperref syntax tests (8 weeks ago)
  * 3113a9a fix: minor error in toc->toggle_help (8 weeks ago)
  * 5c7588e revert: needless fix (ed32deed) (8 weeks ago)
  * ed32dee fix: don't read files unless readable (8 weeks ago)
  * 2b3cbc2 fix: improved 'im' and 'am' text objects (9 weeks ago)
  * f3f5d3a test: more tests for 'im' and 'am' text objects (9 weeks ago)
  * abd461a chore: vim-polyglot now bundles vimtex instead of LaTeX-Box (9 weeks ago)
  *   7137073 merge: note on okular backward search (9 weeks ago)
  |\
  | * 7519038 doc: minor improvement (9 weeks ago)
  | * c009290 revert accidental edit (9 weeks ago)
  | * fd555e6 Add note about backwards search only working from browse mode in Okular (9 weeks ago)
  |/
  * 5ce06b9 fix: use 1 instead of v:true for compatibility (10 weeks ago)
  * 3fe68a0 feat: improve item textobj (10 weeks ago)
  * 0415edb test: more advanced item textobj tests (10 weeks ago)
  * 29f27bf fix: texinputs error with complex filenames (2 months ago)
  * 387b3bc fix: avoid buffer data clash for imap initialization (2 months ago)
  * 7833fb2 feat: add option to specify autoloaded syntax packages (2 months ago)
  * 4adeb3f doc: minor fix (2 months ago)
  *   921cc87 merge: note on forward search + sumatrapdf + win10 (2 months ago)
  |\
  | * f740cba Take out extra line. (2 months ago)
  | * f8b2b31 Run Vim as Admin on Windows for forward-searching (2 months ago)
  |/
  * ecabdb7 feat: allow to disable zathura libsynctex check (3 months ago)
  * 9194c10 feat: allow more control of todos in ToC (3 months ago)
  * 8287981 doc: updated okular settings (3 months ago)
  * f40e5c8 fix: use even more safe shell cmd (3 months ago)
  * 8ab0b0e fix: avoid zsh specific shell syntax (3 months ago)

minimal.vim

call plug#begin('~/.local/share/nvim/data/plugged')
Plug 'lervag/vimtex'
call plug#end()

" Start Vimtex plugin if filetype is .tex
autocmd FileType tex let g:vimtex_enabled = 1

" Set tex flavor to LaTeX, rather than TeX
let g:tex_flavor = 'latex'

" Latexmk settings
let g:vimtex_compiler_latexmk = {
    \ 'backend' : 'nvim',
    \ 'callback' : 1,
    \ 'continuous' : 1,
    \ 'executable' : 'latexmk',
    \ 'options' : ['-verbose',
    \   '-file-line-error',
    \   '-synctex=1',
    \   '-interaction=nonstopmode',
    \ ],
    \}

" Latexmk engines
let g:vimtex_compiler_latexmk_engines = {
    \ '_' : '-pdf',
    \ 'pdflatex' : '-pdf',
    \ 'dvipspdf' : '-dvi -ps -pdfps',
    \ 'xelatex' : '-xelatex -pdfxe',
    \}

minimal.tex

\documentclass{article}
\usepackage[backend=biber,
    bibstyle=authoryear,
    citestyle=authoryear-comp,
    maxcitenames=2,
    minnames=1,
    uniquelist=false,
    bibencoding=utf8]{biblatex}
\addbibresource{/home/lervag/minimal.bib}

\begin{document}

This is a test document.

\cite{

\printbibliography
\end{document}

/home/lervag/minimal.bib

@book { knuth1981,
        author = "Donald E. Knuth",
        title = "Seminumerical Algorithms",
        publisher = "Addison-Wesley",
        year = "1981" }

Commands/Input

Type \cite{ and in insert mode, do CTRL-X CTRL-O.

Observed Behaviour

After updating vimtex Nothing happens. In the statusline, I see -- Omni completion (^O^N^P) Pattern not found

After reverting vimtex update The citation is completed with knuth1981.

Expected Behaviour

Citation is completed with knuth1981 both before and after update of vimtex plugin.

Output from VimtexInfo

System info
  OS: Linux 5.7.8-200.fc32.x86_64
  Vim version: NVIM v0.4.3
  Has clientserver: true
  Servername: /tmp/nvimfLCTTK/0

vimtex project: minimal
  base: minimal.tex
  root: /home/jake/Dropbox/Documents/vimtex_issue
  tex: /home/jake/Dropbox/Documents/vimtex_issue/minimal.tex
  out: /home/jake/Dropbox/Documents/vimtex_issue/minimal.pdf
  log: /home/jake/Dropbox/Documents/vimtex_issue/minimal.log
  aux: /home/jake/Dropbox/Documents/vimtex_issue/minimal.aux
  fls: /home/jake/Dropbox/Documents/vimtex_issue/minimal.fls
  main parser: current file verified
  compiler: latexmk
    backend: nvim
    output: /tmp/nvimfLCTTK/1
    configuration: 
      continuous: 1
      callback: 1
      latexmk options:
        -verbose
        -file-line-error
        -synctex=1
        -interaction=nonstopmode
      latexmk engine: -pdf
    pid: 28854
    cmd: max_print_line=2000 latexmk -verbose -file-line-error -synctex=1 -interaction=nonstopmode -pdf -pvc -e '$pdf_previewer = "xdg-open"' -e '$success_cmd = "echo vimtex_compiler_callback_success"' -e '$failure_cmd = "echo vimtex_compiler_callback_failure"' 'minimal.tex'
  viewer: General
  qf: LaTeX logfile
    config: 
      packages: 
        default: 1
      default: 1
  document class: article
  packages:
    biblatex
    etexcmds
    etoolbox
    geometry
    ifluatex
    ifpdf
    ifthen
    ifvtex
    ifxetex
    infwarerr
    keyval
    kvoptions
    kvsetkeys
    logreq
    ltxcmds
    pdftexcmds
    url

[lervag]

After recently updating vimtex, citation completion no longer works.

Thanks for the detailed report. I've tried to reproduce, but I'm sorry to say I can not. Completion works as expected for me.

minimal.vim

You can simplify the options: No need for the autocommand and no need to specify default settings. The following should suffice:

let g:tex_flavor = 'latex'
let g:vimtex_compiler_latexmk_engines = {
      \ 'dvipspdf' : '-dvi -ps -pdfps',
      \ 'xelatex' : '-xelatex -pdfxe',
      \}

---

Perhaps the problem is related to the cache? Can you try to clear it? There is currently no command, but doing rm ~/.cache/vimtex/*.json should do it (the only drawback is that some things may have a short delay until the cache is reproduced).

[jakewvincent]

You can simplify the options: No need for the autocommand and no need to specify default settings. The following should suffice:

Thank you!

Perhaps the problem is related to the cache? Can you try to clear it? There is currently no command, but doing rm ~/.cache/vimtex/*.json should do it (the only drawback is that some things may have a short delay until the cache is reproduced).

Just cleared the cache---no luck, unfortunately. I forgot to mention this is happening for me on two machines (which share the same init.vim file). Any idea what else I might try?

[lervag]

Could you try bisecting to locate which commit breaks things for you? I.e.:

1. Open a terminal and go to ~/.local/share/nvim/data/plugged/vimtex.

2. First do git pull to get the latest version.

3. In a different terminal, go to the test and check if you can reproduce the error. If not, then I guess it is gone and we stop. If you can reproduce, proceed.

4. Now do git bisect start, then git bisect bad. Then git checkout 8ab0b0e^ (this should be the commit you were on before the :PlugUpdate). Check that the problem is now gone, and if so, do git bisect good. With that, Git should be ready to start the bisect process and will automatically go to the next commit. For each commit, test if the problem is reproducible, and mark the commit with git bisect bad or git bisect good depending on the result.

[j-towns]

Just to say I encountered this issue today but doing git pull in my vimtex directory appears to have fixed it. Unfortunately I don't have time atm to look any further into the cause or which commits completion was failing under, but thought this data point might be useful nonetheless.

[jakewvincent]

1. Open a terminal and go to ~/.local/share/nvim/data/plugged/vimtex.
2. First do git pull to get the latest version.
3. In a different terminal, go to the test and check if you can reproduce the error. If not, then I guess it is gone and we stop. If you can reproduce, proceed.

This didn't fix it; going to step 4.

4. Now do git bisect start, then git bisect bad. Then git checkout 8ab0b0e^ (this should be the commit you were on before the :PlugUpdate). Check that the problem is now gone, and if so, do git bisect good. With that, Git should be ready to start the bisect process and will automatically go to the next commit. For each commit, test if the problem is reproducible, and mark the commit with git bisect bad or git bisect good depending on the result.

After git checkout 8ab0b0e^ the problem was gone. Here's the output at the end of the bisect process:

b7fbff179005845b03b713b7a46fa28784afdf42 is the first bad commit
commit b7fbff179005845b03b713b7a46fa28784afdf42
Author: Karl Yngve Lervåg <karl.yngve+git@gmail.com>
Date:   Sun Jul 12 23:25:40 2020 +0200

    feat: use blg to parse included bib files when possible

    refer: #1745

 autoload/vimtex/complete.vim | 21 +++++++++++++++++++--
 1 file changed, 19 insertions(+), 2 deletions(-)

[lervag]

This is quite as expected. When you say things don't work, can you check if there's a .blg file in your project? Does it work if you remove that file?

[jakewvincent]

Does it work if you remove that file?

Yes, after finishing the bisection search and ensuring citation completion isn't working, once I remove the .blg file, completion works again.

[lervag]

Ok, now we're getting somewhere!

The problem seems to be that, on your end, the .blg method for locating the included bibliography files does not work. To get further, can you please give me:

1. The contents of the .blg file in the case where things don't work.
2. The contents of the preamble in your LaTeX file + the lines that add the bibliographies in your TeX files (e.g. \bibliography{...} or \addbibresource lines).
3. The "structure" of your LaTeX projects (e.g. output of tree or ls -lR).

I think I should be able to figure things out from those things.

[jakewvincent]

1. The contents of the .blg file in the case where things don't work.

[0] Config.pm:304> INFO - This is Biber 2.12
[0] Config.pm:307> INFO - Logfile is 'jwv_syntax_ms.blg'
[18] biber:315> INFO - === Tue Jul 21, 2020, 15:20:40
[32] Biber.pm:371> INFO - Reading 'jwv_syntax_ms.bcf'
[74] Biber.pm:889> INFO - Found 1 citekeys in bib section 0
[84] Biber.pm:4093> INFO - Processing section 0
[93] Biber.pm:4254> INFO - Looking for bibtex format file '/home/jake/Documents/library.bib' for section 0
[129] bibtex.pm:1523> INFO - LaTeX decoding ...
[992] bibtex.pm:1340> INFO - Found BibTeX data source '/home/jake/Documents/library.bib'
[1039] UCollate.pm:68> INFO - Overriding locale 'en-US' defaults 'normalization = NFD' with 'normalization = prenormalized'
[1039] UCollate.pm:68> INFO - Overriding locale 'en-US' defaults 'variable = shifted' with 'variable = non-ignorable'
[1039] Biber.pm:3921> INFO - Sorting list 'nyt/global//global/global' of type 'entry' with template 'nyt' and locale 'en-US'
[1039] Biber.pm:3927> INFO - No sort tailoring available for locale 'en-US'
[1040] bbl.pm:636> INFO - Writing 'jwv_syntax_ms.bbl' with encoding 'UTF-8'
[1041] bbl.pm:739> INFO - Output to jwv_syntax_ms.bbl

2. The contents of the preamble in your LaTeX file + the lines that add the bibliographies in your TeX files (e.g. \bibliography{...} or \addbibresource lines).

Here's the \addbibresource line from the preamble, first:

%-- Bibliography resources --%
\addbibresource{/home/jake/Documents/library.bib}
%\addbibresource{C:/Users/jakew/Documents/library.bib}  % uncomment if on Windows

And the whole preamble:

%! TEX program = dvipspdf

\documentclass[11pt]{article}

%----------------%
%-- Font stuff --%
%----------------%

\usepackage{tinos}
\usepackage{anyfontsize}

%-- Symbols --%
\usepackage{amssymb}
\newcommand{\cmark}{\ding{51}}
\newcommand{\xmark}{\ding{55}}
\usepackage[nointegrals]{wasysym}
\usepackage{marvosym}
\usepackage{textcomp}
\usepackage{upgreek}
\usepackage{pifont}% http://ctan.org/pkg/pifont

%-- Font decoration --%
\usepackage[normalem]{ulem}
\usepackage[outline]{contour} % for putting a glow effect behind text (for head noun boxes)
\newcommand{\myuline}[1]{\uline{\contour{white}{#1}}}

%---------------------%
%-- Page formatting --%
%---------------------%

%-- Margins --%
\usepackage[margin=1in]{geometry}

%-- Columns --%
\usepackage{multicol}

%----------------%
%-- References --%
%----------------%

\usepackage[colorlinks]{hyperref}

\usepackage[backend=biber,
    bibstyle=authoryear,
    citestyle=authoryear-comp,
    maxcitenames=2,
    minnames=1,
    uniquelist=false,
    bibencoding=utf8]{biblatex} % Note that uniquelist=false may lead to citations being ambiguous, esp. if the citation year is the same.

%-- Bibliography resources --%
\addbibresource{/home/jake/Documents/library.bib}
%\addbibresource{C:/Users/jakew/Documents/library.bib}  % uncomment if on Windows

3. The "structure" of your LaTeX projects (e.g. output of tree or ls -lR).

Output of tree:

.
├── alternate
│   └── syntaxtemplate.pdf
├── jwv_syntax_ms.aux
├── jwv_syntax_ms.bbl
├── jwv_syntax_ms.bcf
├── jwv_syntax_ms.blg
├── jwv_syntax_ms.dvi
├── jwv_syntax_ms.fdb_latexmk
├── jwv_syntax_ms.fls
├── jwv_syntax_ms.log
├── jwv_syntax_ms.out
├── jwv_syntax_ms.pdf
├── jwv_syntax_ms.ps
├── jwv_syntax_ms.run.xml
├── jwv_syntax_ms.synctex.gz
├── jwv_syntax_ms.tex
└── outline.xopp

2 directories, 28 files

[lervag]

Ok, I see. In your case, for some reason, you don't get the lines

[135] Utils.pm:75> INFO - Globbing data source '/home/lervag/.vim/bundle/vimtex/test/issues/1751/minimal.bib'
[136] Utils.pm:91> INFO - Globbed data source '/home/lervag/.vim/bundle/vimtex/test/issues/1751/minimal.bib' to /home/lervag/.vim/bundle/vimtex/test/issues/1751/minimal.bib

This explains the problem. I might have been to quick, so I'll add an update that parses things more robust.

[lervag]

Ok, I think I've fixed this now, please test.

[jakewvincent]

Works beautifully. Thanks!! :)

[lervag]

Great, happy to hear it :)