sciunto-org / python-bibtexparser

Bibtex parser for Python 3
https://bibtexparser.readthedocs.io
MIT License
473 stars 132 forks source link

unicode problems #51

Closed beierman closed 7 years ago

beierman commented 9 years ago

I'm having trouble writing the bibtex file (containing unicode) below. I assume the unicode characters fail to write because bibtexparser wants to write ascii, which would work if the unicode would be substituted by the respective latex string. So how do I enforce unicode-to-latex conversion? The correct conversion seems to be in latexenc.py: ("\u2009", "\\hspace{0.167em}"),, but it does not seem to get applied.

Thanks

This code

with open(outfile, 'w') as bibtex_file:
    bibtexparser.dump(bibdb, bibtex_file, writer=writer)

causes this error

Traceback (most recent call last):
[..]
    bibtex_file.write(bibtex_str)
UnicodeEncodeError: 'ascii' codec can't encode character u'\u2009' in position 76: ordinal not in range(128)

on this bibtex file

@article{Mesa-Gresa2013,
abstract = {During a 4-week period half the mice (n = 16) were exposed to EE and the other half (n = 16) remained in a standard environment (SE). Aggr. Behav. 9999:XX-XX, 2013. © 2013 Wiley Periodicals, Inc.},
author = {Mesa-Gresa, Patricia and P\'{e}rez-Martinez, Asunci\'{o}n and Redolat, Rosa},
doi = {10.1002/ab.21481},
file = {:Users/jscholz/Documents/mendeley/Mesa-Gresa, P\'{e}rez-Martinez, Redolat - 2013 - Environmental Enrichment Improves Novel Object Recognition and Enhances Agonistic Behavior.pdf:pdf},
issn = {1098-2337},
journal = {Aggressive behavior},
month = apr,
number = {April},
pages = {269--279},
pmid = {23588702},
title = {{Environmental Enrichment Improves Novel Object Recognition and Enhances Agonistic Behavior in Male Mice.}},
url = {http://www.ncbi.nlm.nih.gov/pubmed/23588702},
volume = {39},
year = {2013}
}
billbrod commented 9 years ago

Not sure if this will be useful to anyone, but I was running into a similar issue and so manually encoded the string as utf8

with open(outfile,'w') as bibtex_file:
    bibtex_str = bibtexparser.dumps(bibdb)
    bibtex_file.write(bibtex_str.encode('utf8'))

And that worked without any issues.

sciunto commented 7 years ago

@Phyks Did you check the status of this issue on master?