Showstopper: citation keys change!

[madhusudansingh]

I am using better-biblatex to access my Zotero collections of journal articles. Mostly it works fine, though duplicate keys are a bummer (the documentation for the alternate JabRef syntax based fields is not adequate).

I am writing to discuss a different, somewhat scary feature of this combination - silent change in item keys.

I was just writing a paper and added a citation to a journal article (call it \cite{Chen2014} for concreteness). While I was writing the conclusions, I found another article, also by a first author named Chen and added it to my Zotero collection.

Using BibDesk, I refreshed the URL that provides the bib file to check the key for the new article. It was also Chen2014. However, to my horror, the citation key to the previous article (the one I had already cited to as \cite{Chen2014}) had changed to Chen2014a!

Now, a typical user is going to have a dynamically growing library. He or she is also going to have a large static set of .tex files (those that have already been written, or the sections of a paper than have been finalized).

This "feature" above means that everytime I have to use a previously written tract of LaTeX, I cannot be certain that article keys have not changed.

If I am not messing up something, this is an utter show-stopper.

Any suggestions?

[retorquere]

Hi, I'm sorry you're experiencing difficulties.

On Jun 24, 2014 11:51 AM, "M Singh" notifications@github.com wrote:

I am using better-biblatex to access my Zotero collections of journal articles. Mostly it works fine, though duplicate keys are a bummer (the documentation for the alternate JabRef syntax based fields is not adequate).

In what ways? Could you please file a separate issue for this?

I am writing to discuss a different, somewhat scary feature of this combination - silent change in item keys.

I was just writing a paper and added a citation to a journal article (call it \cite{Chen2014} for concreteness). While I was writing the conclusions, I found another article, also by a first author named Chen and added it to my Zotero collection.

Using BibDesk, I refreshed the URL that provides the bib file to check the key for the new article. It was also Chen2014. However, to my horror, the citation key to the previous article (the one I had already cited to as \cite{Chen2014}) had changed to Chen2014a!

Now, a typical user is going to have a dynamically growing library. He or she is also going to have a large static set of .tex files (those that have already been written, or the sections of a paper than have been finalized).

This "feature" above means that everytime I have to use a previously written tract of LaTeX, I cannot be certain that article keys have not changed.

If I am not messing up something, this is an utter show-stopper.

I recognize that this is a real issue. The short-term solution would be to generate a bibtex key after importing references. That will prevent the key from being changed after new imports, or editing of the reference, (which would cause similar problems).

For the longer term, I'll try to think of a way to generate stable keys on import, but to be honest, zotero doesn't make this easy. The infrastructure that would allow a clean solution is on the zotero road map but there is no eta on that.

[madhusudansingh]

Thanks for the quick response and confirming that this is a real thing. This means that I (or really anyone who writes a lot of LaTeX documents, and has a growing library of articles) cannot use Better biblatex for any work in the future until this is somehow fixed.

Might I request the following (in addiiton to the above):

1. The format specified in the Zotero settings should actually be used in the drag and drop output. Mine is set to Author:Year. Yet, drag and drop yields \cite{AuthorYear}. Is this a bug?
2. Allow use of Author initials in addition to last name in the citation key (for instance, C. Y. Chen in 2011, could yield, depending on the user-configuration, CYChen2011). This may also reduce the severity of the occurrence of the duplicates.
3. Fix the duplicate issue (autozotbib does) - maybe CYChen2011a, CYChen2011b, etc.

[retorquere]

I've added a setting that allows you to disable conflict resolution. This will yield stable keys, at the cost of potential conflicts (the two entries you mention will get the same key). I have a solution in mind that would structurally fix this, but I'm looking for a non-invasive way to add it to Zotero; it's not something that Zotero handles very gracefully right now.

I don't mean to toot my own horn, but given that this is a blocking issue for many users of LaTeX, you cannot use Zotero, full stop. The default Bib(La)TeX export does the same thing, but has no facility to disable it as BBT now does.

While I usually prefer new issues or feature requests to be filed as separate issues, concerning your questions:

1. (assuming some formatting got wiped away by the github formatter and you meant [Author]:[Year]): that was caused by an perhaps over-zealous key cleaner. I've added ':' to the allowed citekey characters.
2. I'm open to adding this without making it too hard to use. I've mimiced the jabref functions because they seem for the most part to do the right thing at a low cognitive cost. I haven't figured a way yet to do the same for more finegrained field access.
3. BBT does exactly that (although you can now disable it in the preferences). The problem is that Zotero only offers exactly those references that you select to its translators, so there is no way for a translator (such as BBT) to figure out that there is a key conflict unless both happen to be selected -- in which case, the a-b-c prefix thing does happen. The only way to prevent this is from going outside the translator sandbox to always add citation keys to all entries regardless of how they got into Zotero. That can be done, but my first trials slowed down Zotero to an intolerable degree. I have some ideas that might solve it, but this is on my backlog, and I can't give you an estimate for when it will appear.

[retorquere]

The reason autozotbib can btw is that it is not kicked off by interactive user request but internally. so it can choose what items to include in its considerations. BBT ought to work with autozotbib BTW -- autozotbib just allows for automated translations; BBT is one translator (well 4, technically) among the set available.

[madhusudansingh]

Actually, I can use Zotero but its keys (Author_Title_Year) are very long and unwieldy.

Mendeley does a much better job than Zotero on this but then, it is not really free.

[retorquere]

Standard Bib(La)TeX export from Zotero does use longer keys which makes the chance of key collisions smaller but not absent; BBT can be configured to generate the same keys, and you'd have exactly the same chance of key collisions. When key collisions happen for the standard export they get postfixed with numbers instead of letters but trust me, that behavior is exactly the same, and keys will randomly change.

[retorquere]

In what way does Mendeley deal better with this? I could probably create the same behavior.

[madhusudansingh]

Mendeley has inbuilt support for latex citation-keys (they show up in the information pane) and even maintaining sync with an on-disk .bib file (by collection, or full library). In the few years I used Mendeley, I never had any issues with their bibtex support (default is AuthorYear). It just worked out of the box with nearly zero additional configuration. Never saw duplicate keys or key change.

I could be wrong, but I doubt you could do much along those lines. Zotero, for whatever reasons, has accorded less importance to bibtex users than to Word users. For something truly open source, you would have expected the opposite.

Owing to the shoddy bibtex support in Zotero, I have been considering switching back to Mendeley even though I would normally trust Zotero more.

[retorquere]

Ah, they take bibtex seriously. Yes, that will take a change in zotero that has been talked about for a long time but which does not have an indication for an eta. I do have an idea on how it can be made to work regardless, but that requires a bit of care and tinkering to make sure it's stable and not too much of an eyesore. I'm actively working on that, so a first version will probably be available in a few weeks.

[madhusudansingh]

Thanks! I would love to try it out.