Missing UTF-8 encoding for "no-escape" variables

lksnmnn commented 6 years ago

Issue

I tried rendering a variable without escaping HTML but got an error due to missing UTF-8 encoding.

Example

Template: {{& content}} Data: { "content": "<a>My link with Ä german umlaut</a>"} Stack trace

Traceback (most recent call last):
  File "generate_site.py", line 35, in <module>
    partials_ext=PARTIALS_EXTENTION)
  File "/usr/local/lib/python2.7/site-packages/chevron/renderer.py", line 232, in render
    def_ldel=def_ldel, def_rdel=def_rdel)
  File "/usr/local/lib/python2.7/site-packages/chevron/renderer.py", line 206, in render
    output += str(_get_key(key, scopes))

UnicodeEncodeError: 'ascii' codec can't encode character u'\xf6' in position 182: ordinal not in range(128)

Solution

Use UTF-8 encoding for no-escape rendering as well.

PR

See https://github.com/noahmorrison/chevron/pull/29

Also I recommend refactoring the rendering method such that the UTF-8 encoding is done by default. This will remove the duplicate code and prevent similar errors in the future.

Cheers, Lukas

noahmorrison commented 6 years ago

Before I close this, I just want to clarify your PR #29 fixes this issue completely?

lksnmnn commented 6 years ago

If there is no other place in the code where you output a string without transforming it to unicode, this should be it. Works for me and the unit test.

Is there any chance that you can create a new release for the pypi package?

noahmorrison commented 6 years ago

Pypi already includes a 0.12.0 release which includes your PR.

noahmorrison / chevron