citeproc update

[hading]

Bibapp still uses citeproc-0.0.1, which is a very old version that handles very old CSL only (and then still not well).

There is work on more advanced processors, both native ruby and wrapping other processors (I believe there is a very good javascript processor). These will work with CSL 1.0.

It'd be quite desirable to make this update. For display to users we can probably find a way to incorporate the javascript directly, i.e. pass the CSL and bibapp data to the browser and let it run the javascript code to make the display html.

Bibapp may have some internal uses of this as well, and that may be more complex. There is at least one ruby gem that aims to use a javascript interpreter embedded in ruby (for JRuby this is apparently easy, but there seems to be at least one experimental one that is more general) to run the javascript.

[hading]

https://github.com/sstephenson/execjs

is a gem that can use a variety of javascript engines that might be installed (e.g. node.js) to run javascript code from ruby. I'm going to give this a serious look, as it may be a very easy way to get this more up to date.

[hading]

I'm looking at Johnson and citeproc-js. A potential difficulty is that citeproc-js wants you to define some javascript functions and so forth that can get information about the items to go into the bibliography/list of citations. So you kind of load it up with that and then when it needs them it fetches what it needs.

I think that this page:

http://tenderlovemaking.com/2008/04/23/take-it-to-the-limit-one-more-time/

describes how one might be able to do that, but I need to play a bit.

[hading]

Well, except that it might be out of date! I think maybe context => Runtime in the current system.

[hading]

I think this is basically what we want:

context = Johnson::Runtime.new

context['sys'] = {'retrieveItem' => lambda {|x| Work.find(x).title_primary}}

context.evaluate('sys.retrieveItem(1730)') => "Effect of soybean variety and processing on growth performance of young chicks and pigs."

I think I'll try to see if I can incorporate the necessary stuff into one part of Bibapp and then see how things work from there.

[hading]

one may need to load the xmle4x.js or xmldom.js in the src directory of the citeproc-js distribution before loading citeproc.js.

See http://gsl-nagoya-u.net/http/pub/citeproc-doc.html for citeproc.js usage info.

I'm not sure that this will load correctly into Johnson, so I may need to look at some other stuff.

[hading]

Get a little farther using 'therubyracer' gem.

context = V8::Context.new

context.load('/home/hading/repos/citeproc-js/src/xmldom.js')

context.load('/home/hading/repos/citeproc-js/citeproc.js')

I think this loads the necessary stuff from citeproc-js. We then still need to hook things up as described above, make the csls available somehow, etc. But I think this loads the core code.

[hading]

https://github.com/citation-style-language/schema has information describing what the input to a processor should look like. I think. So we'll need to make Work conform to this. Probably best to do a very rough cut first to make sure the CSL transformer is working and then refine the mapping from Work to this.

[hading]

No dice yet. I get an error in the javascript when trying to instantiate a processor, and I'm not sure how to get around it, or even if I can. Will think some more, but I don't know.

[hading]

Well, I got a version working using jruby and therubyrhino. I can generate csl-entry divs, although it's still not clear how useful they are. It's probably just that I don't know what I'm doing, though.

If it works out we could potentially include a jruby jar with bibapp and have it start, load up this code, and use event machine or something to listen on a port and then processes for citations as needed. That's an idea, anyway. It could also be a web service without much problem, but I think that would detract from the self-containedness of Bibapp.

[hading]

Put in some more time on this, still can't get it to work completely. I have trouble with some of the more complex data structures (e.g. attaching an author to an item). Though I'm following the pattern found in the tests there is an error thrown along the way when the javascript code tries to process, and it's wholly unclear what is happening.

I can't get things to work using raw js in rhino, so I'm not all that optimistic here.

[hading]

This is interesting:

http://www.zotero.org/support/dev/citation_styles/citeproc-node

tells (maybe) how to set up a node.js based citeproc server. I'm going to try it.

[hading]

Well, those instructions are a little rough, and after overcoming a few problems it gets to the point where it just seg faults.

[hading]

The segfault appears to be because of the version of node:

https://groups.google.com/forum/?fromgroups#!topic/zotero-dev/HZ-bw_7Bwxo

Maybe check back later to see if citeproc-node gets updated for current versions of node.

[hading]

In short, I continue to think there's promise with an approach along one of these lines once one of them works easily, but that doesn't seem to be the case yet.