antocuni / capnpy

Other
45 stars 26 forks source link

`repr()` raises `UnicodeEncodeError` on Python 2.7 if text fields contain non-ASCII characters #46

Open nickmyatt opened 4 years ago

nickmyatt commented 4 years ago
$ cat example.capnp 
@0xa6efb970dc0d109b;
struct Foo {
    bar @0 :Text;
}
$ cat example.py 
# -*- coding: utf-8 -*-

import capnpy

example = capnpy.load_schema('example')

foo = example.Foo(bar='£')

repr(foo)
$ python example.py
Traceback (most recent call last):
  File "example.py", line 9, in <module>
    repr(foo)
UnicodeEncodeError: 'ascii' codec can't encode character u'\xa3' in position 14: ordinal not in range(128)
$ python --version
Python 2.7.14
nickmyatt commented 4 years ago

This occurs when schemas are AOT compiled with text_type = "bytes". I don't know what text_type defaults to for dynamic compilation using load_schema?

I suspect this is the cause: https://github.com/antocuni/capnpy/commit/ee0edc647429a7ae5739b2c1271c4703a52e14f2