Problems with umlauts in author names

[gladwig]

Hi,

I tried formatting some BibTeX files with bibtex-ruby and citeproc-ruby and encountered a problem when author names contain umlauts (ä, ö, ü). The problem seems to lie with citeproc-ruby (note the family name with an umlaut):

book = {
        'author' => [{ 'given' => 'Edgar Allen', 'family' => 'Müller' }],
        'title' => 'Poetry, Tales, and Selected Essays',
        'type' => 'book',
        'issued' => { 'date-parts' => [[1996]] },
        'editor' => [{ 'family' => 'Quinn', 'given' => 'Patrick F.'}, { 'family' => 'Thompson', 'given' => 'G.R.' }],
        'publisher' => 'Library of America',
        'publisher-place' => 'New York'
    }

>> CiteProc.process(book)
=> "M\303\274llerEdgar. (1996). Poetry, Tales, and Selected Essays.  (P. F. Quinn & G. R. Thompson, Eds., , Tran.). New York: Library of America."

As you can see the family and given name are mangled. Without the umlaut everything is fine:

book = {
  'author' => [{ 'given' => 'Edgar Allen', 'family' => 'Mueller' }],
  'title' => 'Poetry, Tales, and Selected Essays',
  'type' => 'book',
  'issued' => { 'date-parts' => [[1996]] },
  'editor' => [{ 'family' => 'Quinn', 'given' => 'Patrick F.'}, { 'family' => 'Thompson', 'given' => 'G.R.' }],
  'publisher' => 'Library of America',
  'publisher-place' => 'New York'
}

>> CiteProc.process(book)
=> "Mueller, E. A. (1996). Poetry, Tales, and Selected Essays.  (P. F. Quinn & G. R. Thompson, Eds., , Tran.). New York: Library of America."

[inukshuk]

Which Ruby version did you use for the tests? I just ran this on 1.9.3 and the umlaut works as expected.

[gladwig]

Ah, sorry for not mentioning that. This the Mac OS X Lion default, I think:

ruby 1.8.7 (2010-01-10 patchlevel 249) [universal-darwin11.0]

[inukshuk]

Yes, it's very likely that there are still unicode issues in 1.8. Do you have the active support gem installed? That may help. Otherwise, there is not much we can do currently, because citeproc-ruby is in the middle of a rewrite. However, there are options:

• you could install ruby 1.9.2 or 1.9.3 (I would recommend you to install via rvm or rbenv)
• you could use the javascript engine which is part of the rewrite and should work quite well on 1.8. If you want to do that, you need to uninstall citeproc-ruby and then install the gems citeproc and citeproc-js; additionally you need to install the johnson gem (javascript runtime). The results from the processor will look a little bit different, but the upside is that the citeproc-js is pretty much feature complete.