Commas in Bib(La)Tex keywords field

[oddious]

First of all, thanks for a great piece of work! I'm currently adopting Better Biblatex into my workflow and it would be really great if there were an option to use commas in keywords, while using ', ' (comma+space) as a keyword separator, as is realized by the default Bib(La)Tex translators, e.g.,

... keywords = {1,2-diol, propargyl alcohol, reaction of, triphosgene}, ...

Update: after a short survey it seems to me that different software expects different format, and Better BibTex attempts to stay on the safe side. However, a possibility to set up the delimiter and the allowed symbols, maybe as a hidden about:config option would be greatly appreciated.

[retorquere]

There is indeed not any standard on how keywords should be encoded, or how they should be interpreted. Allowing user-specified separators would mean I'd also have to find a way to escape them; I agree the current approach is ham-fisted, but it guarantees valid, processable BibTeX output. I had not anticipated chemical-compound keywords however, where the commas are meaningful.

I'll think about something that supports your use-case; perhaps always add , (doesn't hurt) and an option to scrub commas in some way. If you're in a hurry, and not scared of a little javascript, something like the following postscript ought to do the trick (untested):

if (this.tags) {
  var tags = [];
  for (var i = 0; i < this.tags.length; i++) {
    tags.push(this.tags[i].tag);
  }
  // the "replace" param overwrites the existing keywords field.
  this.add({ name: 'keywords', replace: true, value: tags, sep: ', ' });
}

[oddious]

Thanks a lot, I'll test the code. I didn't know about the postscript feature, seems to be very powerfull. I tried to dig into the sources and found the relevant sections in the translators and reference.js, but for some reason my "patches" didn't have any effect. And yeah, a lot of javascript scares me a little bit. With the postscript, however, I might be even curious of hacking this and that. The real muddle will come with importing I guess. Wouldn't it be enough in this case just to set the delimiter and don't think about any special characters except really latex-troublesome, i.e. why to bother with commas if the delimiter is, say, ; and vice versa? And if not then, well, it is the responsibility of the user to take care of their keywords or rather to choose an adequate delimiter to suit their special needs. Anyway for me imporing is irrelevant until there's an automatic bidirectional sync between Better BibTex and a bib file. By the way are you interested in such a feature request?

[retorquere]

Postscript is fairly new, and I don't really advertise it much, as it's a little tricky to get right. If something useful crops up, I usually just build it into BBT. The reason your postscript didn't take will be in the debug log (preferences -> advanced -> enable, try the export again, preferences -> advanced -> view output).

For output, this.add will take care of encoding LaTeX characters properly, and you may feel free to use ; instead of ,.

Importing is a different matter though. I can do pretty much what I want in the exporter as I can be sure it always runs in Zotero, but the importer also gets sent to other browsers through the chrome and other browser plugins, and I can't access the guts of my extension there, so importing is more constrained.

I'm not principally opposed to bidi sync, but I wouldn't expect it anytime soon, for several reasons:

• Zotero wants to put files it imports into a new folder, so I'd have to re-implement chunks of Zotero to bypass that so it'd import into an existing folder
• I'd have to duplicate the stuff in Zotero that does sync collision handling for changes in both ends (collision handling is tricky stuff)
• There's not really a good way to retain state in a BibTeX file; if an entry is missing in the BibTeX file but present in Zotero, does that mean it needs to be added to the BibTeX, or deleted from Zotero?
• Probably more stuff I haven't considered. Sync is tricky.

What's the use-case, if I may ask?

[retorquere]

Ah, I think I've found it; the script should have been

if (this.item.tags) {
  var tags = [];
  for (var i = 0; i < this.item.tags.length; i++) {
    tags.push(this.item.tags[i].tag);
  }
  this.add({ name: 'keywords', replace: true, value: tags, sep: ', ' });
}

tags are defined on the item being exported (this.item), not on the reference being constructed (this)

[oddious]

I herein confirm that this piece works as supposed. Thank you very much!

Regarding my use-case. I've been using Zotero to grab pdfs from publishers and bibliorgaphy (mainly as ris) from databases. My planning and note-taking workflow, however, is tightly tied to Emacs' Org-mode. I would also like to move the majority of my writing tasks from office suites into Latex and thus Emacs/Org-mode as well. That's where Better Bibtex apparently comes from. In Emacs, bibtex files can conveniently be managed with ebib. The latter is even capable of batch processing, the feature that Zotero lacks, for an unclear reason. For me thus it would be very convenient to have a possibility of syncing the changes, mainly keywords, back to Zotero. Right now I actually cannot edit anything outside Zotero as it will be instantly overwritten by autoexport. I can of course give up autoexporting, but then very soon my databases will diverge, something I intuitively do not want to happen. I realize though that the described set up must be rare and most people are quite satisfied with managing their references in Zotero and exporting into bibtex just before using.

[retorquere]

What kind of batch processing does ebib do?

[oddious]

Eh, what I actually meant is batch editing. It appears not that much though, only adding keywords to multiple entries at once, I thought it can do more. However, this is 80% of what I expect from batch editing, the rest 20% being the possibility to untag multiple entries. Mendeley is a superior competitor here, as far as I remember.

[retorquere]

I have only used mendeley briefly, so I'm not in a position to comment.

[oddious]

It was a breath of fresh air for me 5 or 6 years ago, but there're certain important things which Zotero does better, not speaking of openness, of course.

[retorquere]

The openness matters to me as a point of pragmatical interest. I haven't yet met a piece of software that does what I want the exact way I want it. Extensible or open source makes that I am not beholden to the programmers' interest. I might be fine with Mendeley if they had at least a minimal extension API, but they don't, so the things they do better are eclipsed by the fact that I can't tweak them where I disagree (I am very opinionated on how software should work for me rather than the other way around). Same goes for Scrivener; seems like an interesting program, and I was gifted a license, but non-adaptability makes it a non-starter for me, and I'm not going to lock up my work in a program I can't tinker with.

There's plenty of warts on Zotero, some of which I have addressed, some of which I really want to have addressed, but for me it's simply the best of a bad lot (the bad lot being software in general).

[retorquere]

Looks like bulk tagging is supposed to be supported: http://ideophone.org/12-zotero-tips-and-techniques/

[oddious]

Completely agree with you about open-source, that's why we're all here. And thanks a lot for the tips!

[retorquere]

I've made some minor changes that will make your postscript simpler -- per 1.5.3, the whole script would (will have to) be

this.add({ name: 'keywords', replace: true, value: item.tags, sep: ', ' });

so just this single line.

I'm not strictly ideological about open source. I have a soft preference for it, but probably because I'm a tinkerer. I use both Linux and OSX; I like the macbook air for its battery performance/weight balance, and OSX allows for sufficient tinkering (although I'm still stumped why people are so enthusiastic about OSX on the whole -- I don't find it to be better than Windows or Ubuntu), and I like Linux (currently in its Ubuntu form) because I'm used to it. I'd like windows better if cygwin were a better integrated experience (again because I'm used to Unixoids), but on the whole it's on a par with OSX qua tinkerability these days, and sometimes better than OSX.

[oddious]

Thank you, I think this solution is googlable now and hope it will be also helpful for other people.

Windows has had huge problems with their update system, at least in versions 6 and 7. After some amount of updates had been accumulated, the whole system began to experience considerable slowdown which worsened with time. Maybe this issue is partially fixed in the latest versions, I hardly care as I see no need to switch to windows unless I'm forced to. The latter is not impossible though, because there's still a lot of special software (for example in chemistry) which hasn't be ported/doesn't have usable analogues outside Windows. Currently wine is doing the job for me. As for now, I do not see any reasons why I may switch to OSX either. My mobility is provided by an Android tablet with a hard keybord (total ~600 g) ssh-ing through a vpn to my rock-solid Debian laptop located at my institution. This way I can do most of my non-graphical tasks and graphical tasks are usually work-related so in that case I'd better stay longer at work rather than taking them home or, worse, somewhere else.

Ok, seems like a little bit too much for a single comma. Thank you for your efforts once again!

[retorquere]

Always a pleasure.

I have added your sample to the postscript wiki just to make sure.

I am too often without network connectivity (despite the claims Vodafone makes in this regard), so a remote solution won't do for me. I have an Ubuntu home server I could use (and indeed often did in the past), but having stuff at hand works much better for me, certainly where I need to test Zotero stuff.

I've never had great difficulties with Windows (which version is 6? I know 7), but as it's usually just the environment I use for Chrome/Firefox/Putty/Cygwin, it doesn't really accumulate cruft. I used to care about what people had on their desktops, but that was in younger days.

The sole reason I have OSX is that it came with what was at time of purchase the most bang for the buck when looking for a light system with long battery. The other option was a surface pro, which is actually a decent system, sans the keyboard that you can get for it, which is crap (I had one for about 3 months). I'd run Linux or Windows on the macbook air if I could, but it is terrible for battery life, which was the whole point of the purchase.

[oddious]

Vista was the 6th version. Windows slowdown with time is intrinsic to the system, here it's explained in more detail http://superuser.com/questions/890038/why-is-checking-windows-update-so-slow, the answer therein applies to the whole windows experience, not only to checking for updates. Although the actual experience will apparently depend on the hardware, and the external programs will also matter to some extent.

There's perhaps no ultimate solution suitable for everyone, and one therefore chooses what suits their needs best. It's only since very recently that good alternatives to Apple laptops are available.

[retorquere]

I know a little about how Windows works, and while that gives a plausible explanation of how windows update works, it is not obvious to me how that ties into the general workings of windows from a technical point of view.