More sophisticated citekey completion

[dsanson]

Vim allows completion functions to return a list of dictionaries, rather than just a list words. The simplest format is

{'word': 'blah', 'menu': 'this is blah'}

So, for our purposes, it might be:

{'word': 'geach1972', 'menu': 'Geach, Logic Matters'}

were the value of 'word' is the citekey that will be inserted, and the value of 'menu' is what shows up in the popup menu.

In order to implement this, I need smarter parsing of the supported bibliography files. This can either be implemented directly via regexs, or we can lean on existing parsers, if they are available. The advantage of using regexes is that it is light-weight---we avoid introducing new dependencies---and they probably work fine.

Another option---inspired by vim.latex-box's use of latex+bibtex to solve a similar problem---would be to collect matching keys and then use pandoc to generate a plaintext bibliography and then parse that. The trouble is that the usual CSL styles don't include the citekey. It might not be too hard to generate a custom CSL file for this purpose. But the process is probably too slow for something like completion.

[dsanson]

For bibtex, it looks like I might be able to implement this using the bibtex-ruby gem. I can also make it search not just for citekeys but for any keywords, so that

@rabbit

will suggest any articles containing the word 'rabbit' in any fields. The parsing is easy. I still struggle getting results in ruby back to vimscript.

[fmoralesc]

Check the changes I've made on the multibibs branch: I'm supporting the dictionary format for completion.

I've rewritten Pandoc_bibkey in python, but that was just because I'm more used to it, and can be rewritten in ruby again if needed. (Actually, the key scanning was much cleaner in ruby. Damn python's lack of non-fixed regex look behinds!)

EDIT: I found a way to simplify the regexes. The python code should be much clearer now.

[dsanson]

Grrr! You are pythonifiying everything. I'm much more comfortable in ruby. Of course, you're also making everything better, so I can't really complain ;-)

[fmoralesc]

Sorry for that! ;) But you were rubyfying everything first!

[fmoralesc]

BTW: what do you think is best for sorting the completion items: sorting by key or by title?

[fmoralesc]

I think I've tested the new features enough. If there are no comments, I'll merge the multibibs branch into master.

[dsanson]

Not working for me. When I try to complete a key, vim hangs for several seconds, then dumps a load of error messages.

As for sorting: I would support sorting by citekey, not title. Typically when I want to enter a citation, I know the author for certain, know the title more or less, and may or may not remember the year (and my citekeys are authoryearxx). And if we sort by title, then we have to worry about whether or not to ignore leading articles like 'the' and 'an' and the like.

[fmoralesc]

OK about the sorting part.

That error seems to be a problem with pybtex. What version do you have? I have 0.15 here.

[fmoralesc]

I pushed some changes, so the plugin will resort to the fallback procedure if pybtex fails. Can you check if it solves the problem?

[dsanson]

pybtex 0.15. I just installed it via pip, and haven't tried to use it aside from this. So I'll test that and get back to you.

As for the changes: yes, it no longer throws a bunch of errors. But it still hangs (for about 7 seconds) before it gives up and provides the fallback completion. And it does this every time. (By contrast, when I didn't have pybtex installed at all, on the previous version, it immediately completed via the regex.)

[dsanson]

Turns out I had a problem with my bibtex file: an unquoted journal title. So now it works, but it is slow. 7 seconds before the matches pop up. To be fair, I have a little over 1000 items in my database, but this is too slow to be useable.

I noticed that you are putting every matched bibfile into b:pandoc_bibfiles. And you are putting every bibfile in the working directory in there too. I don't think this is the right way to go. For one thing, it will only slow things down more (though my 7 seconds is after manually setting b:pandoc_bibfiles to only include the one file). But also, it gives me too little control. If I have a bib file with the same file name in my directory, that's probably the file I want. If I don't have a bibfile with the same file name the document directory, but I do have some other bibfile in that directory, that's probably the file I want. And if I've put a different bibfile at .pandoc/default.bib, that probably means I want to use it instead of the one in my texmf folder.

I tried completions just against the sample bibtex file from the link above, and they didn't all work. I've gisted a modified version of that file here. It looks to me that the problem is that you aren't matching citekeys that start with an uppercase letter, like

Zurek:1993
Primes

The others seem to work.

Also, can we get 'menu': ',