hplgit / doconce

Lightweight markup language - document once, include anywhere
http://hplgit.github.io/doconce/doc/web/index.html
Other
309 stars 60 forks source link

Python 3 encoding problems with latest doconce version #145

Closed reckoner closed 6 years ago

reckoner commented 6 years ago

With respect to Python 3.6, it seems like pkg_resources doesn't know how to open the utf8 doconce.py file:

   File "/opt/conda/bin/doconce", line 4, in <module>
     __import__('pkg_resources').run_script('DocOnce==1.4.4', 'doconce')
   File "/opt/conda/lib/python3.6/site-packages/pkg_resources/__init__.py", line 748, in run_script
     self.require(requires)[0].run_script(script_name, ns)
   File "/opt/conda/lib/python3.6/site-packages/pkg_resources/__init__.py", line 1515, in run_script
     source = open(script_filename).read()
   File "/opt/conda/lib/python3.6/encodings/ascii.py", line 26, in decode
     return codecs.ascii_decode(input, self.errors)[0]
 UnicodeDecodeError: 'ascii' codec can't decode byte 0xe2 in position 26805: ordinal not in range(128)

I can make this go away by forcing pkg_resources to open the file with the utf8 encoding but I was wondering if there is another more support-able way to do this.

KGHustad commented 6 years ago

I can't reproduce this. Which OS are you using, and how did you install DocOnce?

reckoner commented 6 years ago

When I do:

import locale
locale.getpreferredencoding()

I get the following:

'ANSI_X3.4-1968'

I'm betting that you get the UTF-8 output?

reckoner commented 6 years ago

This is fixable with the following environment variable:

export LANG=C.UTF-8
KGHustad commented 6 years ago

Yes, I get UTF-8, since my LANG is set to en_GB.UTF-8. DocOnce users should ensure that the locale environment variables (LC_ALL, LANG, etc.) are set to some value on the form *.UTF-8.