Global Bib File

[benbrastmckie]

Although I can get bib completion to work with a local .bib file, I cannot manage to get the same to work for a global .bib ~/texmf/bibtex/bib/Zotero.bib which I source with \bibliography{Zotero} at the end of my latex document. I use this global .bib file for everything and then have a keybinding set up to generate a local .bib file which only includes entries for the citations that occur in the local latex document once it is finished.

I am wondering: how can I get cmp-vimtex to find my global .bib file ~/texmf/bibtex/bib/Zotero.bib?

Thanks so much for an excellent plugin/extension. I'm so pleased to have it back up and running :)

[micangl]

Thanks for reporting. Could you post the output of the files variable?

vim.fn["vimtex#paths#pushd"](vim.b.vimtex.root)
local files = vim.fn["vimtex#bib#files"]()
vim.fn["vimtex#paths#popd"]()

Before running them, open that file with Neovim and wait 20 seconds, just to make sure that every potential asynchronous parsing has ended.

Thanks so much for an excellent plugin/extension. I'm so pleased to have it back up and running :)

Glad to hear it! How are finding the new bibliographic search functionality? Do you have any suggestions to improve it?

[benbrastmckie]

If I am correct in checking the variables with :lua vim.print( ... ) as well as :lua print(vim.inspect( ... )), I get the following:

:lua vim.print(vim.fn["vimtex#paths#pushd"](vim.b.vimtex.root))  YIELDS: 0
:lua vim.print(local files = vim.fn["vimtex#bib#files"]())  YIELDS: {}
:lua vim.print(vim.fn["vimtex#paths#popd"]())  YIELDS: E5108: Error executing lua function vimtex#paths#propd, ...

Perhaps also worth mentioning, :Telescope bibtex works just fine. If I have a local bib file sourced, it will find the bib data included there, where my whole library shows up if I source the global bib file. Note that I do not have to build the latex file for this to be true since Telescope bibtex simply knows where to look given which bib file is sources (the latex file does obviously need to be saved). These are the relevant lines from my latex document, only one of which I will include at any given time:

\bibliography{Zotero} %% global bib file found in ~/texmf/bibtex/bib/
\bibliography{PaperName} %% local bib file

The same is not true for cmp-vimtex which seems to require the document to be built in order to find the bib data. Say that I have only loaded the global bib file and build the latex document. As already mentioned, cmp-vimtex does not find the bib data. Then suppose I load a local bib file instead. Still no dice. But if I rebuild the document, then cmp-vimtex can find the local bib data. Similarly, if I then turn off vimtex's build-mode and load the global bib file only, cmp-vimtex continues to find the local bib data (assuming the file still exists). Of course, once I rebuild the document, cmp-vimtex is no longer able to find the local bib data. By contrast, Telescope bibtex doesn't care whether the document has been built or not, looking up whatever bib file is sourced in the saved but possibly unbuilt latex file.

Now for the relevant parts of the nvim-cmp.lua config file:

return {
  "hrsh7th/nvim-cmp",
  event = { "InsertEnter", "CmdlineEnter" },
  dependencies = {
    "hrsh7th/cmp-buffer", -- source for text in buffer
    "micangl/cmp-vimtex",
    -- Other dependencies...
  },
  config = function()
    local cmp = require("cmp")

    cmp.setup({
      -- formating for autocompletion
      formatting = {
        fields = { "kind", "abbr", "menu" },
        format = function(entry, vim_item)
          vim_item.kind = string.format("%s", kind_icons[vim_item.kind]) -- Kind icons
          vim_item.menu = ({
            vimtex = (vim_item.menu ~= nil and vim_item.menu or ""),
            buffer = "[Buffer]",
            -- Other formatting for sources...
          })[entry.source.name]
          return vim_item
        end,
      },
      -- sources for autocompletion
      sources = cmp.config.sources({
        { name = "vimtex" },
        { name = "buffer", keyword_length = 3 },
        -- Other sources...
      }),
    })

  end,
}

As for the vimtex-cmp.lua config file, I have:

return {
  "micangl/cmp-vimtex",
  config = function()
    require('cmp_vimtex').setup({
      additional_information = {
        info_in_menu = true,
        info_in_window = true,
        info_max_length = 60,
        match_against_info = true,
        symbols_in_menu = true,
      },
      bibtex_parser = {
        enabled = true,
      },
    })
  end,
}

And for the vimtex.lua config file, I have:

return {
  "lervag/vimtex",
  dependencies = {
    "micangl/cmp-vimtex",
  },
  version = "*",
  event = { "BufReadPre", "BufNewFile" },
  config = function()
    vim.g['vimtex_view_method'] = 'zathura'
    vim.g['vimtex_quickfix_mode'] = 0
    vim.g['vimtex_mappings_enabled'] = 0 -- Ignore mappings
    vim.g['vimtex_indent_enabled'] = 0 -- Auto Indent
    vim.g['vimtex_syntax_enabled'] = 1 -- Syntax highlighting
    vim.g['vimtex_log_ignore'] = ({
      'Underfull',
      'Overfull',
      'specifier changed to',
      'Token not allowed in a PDF string',
    }) -- Error suppression
    vim.g['vimtex_context_pdf_viewer'] = 'okular'
  end,
}

If there is any other information that I can provide, please don't hesitate to let me know.

Regarding suggestions to improve the bib search functionality, it's honestly pretty great. I'll have a think about what could improve it further, but so far I'm psyched.

[micangl]

Sorry if it took me a while to get back to you. I hoped this problem would be quicker to solve. Unfortunately, I'm quite swamped because of some university exams, and I'll be until the 5th of February; after that date, I'll be able to look into this.

Sorry for the incovenience.

[benbrastmckie]

Not a problem. Good luck with your term!

[micangl]

I'm finally taking a deeper look into this issue.

You seem to have set a global .bib file for the bibtex extension of Telescope. From what I gathered, this means that you should always be able to search in this file, even if it hasn't been explicitly sourced.

Second thing, I can't understand how the latex compiler knows where to look for the .bib file, when you include it with \bibliography{Zotero}.

[benbrastmckie]

Hey no worries! Hope your term went well.

Yes, I have a global .bib set and this is always accessible via Telescope.

I am not sure how latex knows where to look, but it seems that it looks in both the /texmf/bibtex/bib/ directory and the local project directory for the file specified by \bibliography{ } if no path is given. Could cmp-vimtex emulate that behavior?

[micangl]

Ok, now I'm starting to understand this better.

It seems that vimtex doesn't check for files under ~/Library/texmf/bibtex/bib. I say this because the commands you issued earlier produced no output. If I'm correct, the normal omnifunc completion shouldn't work either; it's invoked with ctrl-x rapidly followed by ctrl-o (if it does work, please post a screenshot).

If it does work it may be because the commands you issued earlier were incorrect; this may be better:

:lua vim.print(vim.fn["vimtex#paths#pushd"](vim.b.vimtex.root))
:lua vim.print(vim.fn["vimtex#bib#files"]()) 
:lua vim.print(vim.fn["vimtex#paths#popd"]())

Does kpsewhich find the file?

[benbrastmckie]

I couldn't get ctrl-x followed by ctrl-o to do anything besides move to the old position but did get some output from the commands you provided when \bibliography{Counterfactuals} was included, specifying a local .bib file:

0 FROM :lua vim.print(vim.fn["vimtex#paths#pushd"](vim.b.vimtex.root))
{ "Counterfactuals.bib" } FROM :lua vim.print(vim.fn["vimtex#bib#files"]()) 
0 FROM :lua vim.print(vim.fn["vimtex#paths#popd"]())

When I switched to \bibliography{Zotero} which sources my global the .bib file, the second command was reduced to the following where the others were the same:

{} FROM :lua vim.print(vim.fn["vimtex#bib#files"]()) 

I tried running the following from the terminal but not sure if this is what you intended:

benjamin@nandi ~> kpsewhich ~/texmf/bibtex/bib/Zotero.bib
/home/benjamin/texmf/bibtex/bib/Zotero.bib
benjamin@nandi ~> 

Let me know if there is any other information that I can provide.

[micangl]

Sorry, I should have specified that I meant the output of

kpsewhich Zotero

and that of

kpsewhich Zotero.bib

[benbrastmckie]

Got it. Here's the output:

benjamin@nandi ~> kpsewhich Zotero 
benjamin@nandi ~ [1]> kpsewhich Zotero.bib
/home/benjamin/texmf/bibtex/bib/Zotero.bib
benjamin@nandi ~> 

So looks like it can find the file running the second command.

[micangl]

I'm thinking this must be related to the way Vimtex looks for .bib files. Said hypothesis was advanced by @lervag in https://github.com/lervag/vimtex/issues/2596#issuecomment-1382433102

[lervag]

@benbrastmckie

And for the vimtex.lua config file, I have: …

Please, don't lazy load VimTeX! Do not use event = { ... }. Doing this does not give any benefit and it breaks the inverse search feature which uses a global plugin. In case you've configured lazy.nvim to use lazy = true by default, you may also need to add lazy = false.

Further, VimTeX does not depend on cmp-vimtex; it is the other way around. Also, I would advice to use init instead of config. I've proposed an update for your config here, where I also removed some

return {
  "lervag/vimtex",
  init = function()
    vim.g.vimtex_view_method = 'zathura'
    vim.g.vimtex_quickfix_mode = 0
    vim.g.vimtex_mappings_enabled = 0 -- Ignore mappings
    vim.g.vimtex_indent_enabled = 0 -- Auto Indent
    vim.g.vimtex_log_ignore = ({
      'Underfull',
      'Overfull',
      'specifier changed to',
      'Token not allowed in a PDF string',
    }) -- Error suppression
    vim.g.vimtex_context_pdf_viewer = 'okular'
  end,
}

[lervag]

Got it. Here's the output:

benjamin@nandi ~> kpsewhich Zotero 
benjamin@nandi ~ [1]> kpsewhich Zotero.bib
/home/benjamin/texmf/bibtex/bib/Zotero.bib
benjamin@nandi ~> 

So looks like it can find the file running the second command.

I'm thinking this must be related to the way Vimtex looks for .bib files.

VimTeX does use kpsewhich to find bib files. @benbrastmckie Does omnicompletion work; i.e. if you write \cite{<c-x><c-o>, does it provide the expected completion candidates?

[benbrastmckie]

Thanks for the help! I managed to git it working. I cleaned up my vimtex.lua file as suggested but was still having the same problem. I couldn't get \cite{<c-x><c-o>} to work for my global bib, but could for other smaller bib files in the same global directory. I then tried cutting and pasting my entire library into another differently named global bib file and that seemed to work. So in the end, all I had to do was export my library under a new name. I am not sure why this seems to make the difference, but thought I would see if you had any sense. In any case, I'm very happy to have autocomplete up and running.

Thanks again!

[lervag]

Thanks again!

No problem!

I then tried cutting and pasting my entire library into another differently named global bib file and that seemed to work. So in the end, all I had to do was export my library under a new name. I am not sure why this seems to make the difference, but thought I would see if you had any sense. In any case, I'm very happy to have autocomplete up and running.

That's strange and doesn't make much sense to me either. I've found kpsewhich to be useful to investigate things like this, though. But another thing that might explain it is the VimTeX cache. You could try clearing it with :VimtexClearCache ALL. I don't immediately understand why, but I've found I often make some mistakes with the cache, so there may be a bug I'm not aware of.

[benbrastmckie]

That fixed it! Just to be sure, I renamed the file I had named 'Library.bib' back to 'Zotero.bib' and this reproduced the error. Running :VimtexClearCache ALL fixed the problem, so now autocompletion works as before. Thanks so much for the tip! Is there something I am doing wrong to cause the problem? I don't mind running that command as needed if it resurfaces.

[lervag]

That fixed it! Just to be sure, I renamed the file I had named 'Library.bib' back to 'Zotero.bib' and this reproduced the error. Running :VimtexClearCache ALL fixed the problem, so now autocompletion works as before.

Great, glad to hear it!

Thanks so much for the tip! Is there something I am doing wrong to cause the problem? I don't mind running that command as needed if it resurfaces.

Not sure. It's a little bit hard to say. I hope this is a rare and not important bug. Let me know if you have the same happening again, it would be interesting to inspect the cache file before you clear it.

[micangl]

Very well, since it seems like the problem is gone, I will close the issue.

[benbrastmckie]

Hopefully it won't come back, but if it does, I'll try to find that Cache file. Thanks again!