pyopenapi / pyswagger

An OpenAPI (fka Swagger) client & converter in python, which is type-safe, dynamic, spec-compliant.
MIT License
385 stars 89 forks source link

Unicode strings with primitive _str #89

Closed swen-edk2 closed 8 years ago

swen-edk2 commented 8 years ago

I get an UnicodeDecodeError when parsing strings encoded in utf-8 with pyswagger. Shouldn't a value specified with "type: string" containing unicode characters be parsable by _str?

File "/home/swen/style/web/apitest/test_styles.py", line 18, in test_get_styles_logged_in response = self.client.request(self.api_app.op"dropastyle.api.objects.styles_get") File "/home/swen/style/web/local/venv/local/lib/python2.7/site-packages/pyswagger/contrib/client/requests.py", line 69, in request raw=six.BytesIO(rs.content).getvalue() File "/home/swen/style/web/local/venv/local/lib/python2.7/site-packages/pyswagger/io.py", line 371, in apply_with self.data = r.schema.prim(data, self.op._prim_factory, ctx=dict(read=True)) File "/home/swen/style/web/local/venv/local/lib/python2.7/site-packages/pyswagger/spec/v2_0/objects.py", line 92, in prim return prim_factory.produce(self, v, ctx) File "/home/swen/style/web/local/venv/local/lib/python2.7/site-packages/pyswagger/primitives/init.py", line 182, in produce val = _2nd(obj, ret, val, ctx) File "/home/swen/style/web/local/venv/local/lib/python2.7/site-packages/pyswagger/primitives/comm.py", line 40, in _2nd_pass_obj return ret.apply_with(obj, val, ctx) File "/home/swen/style/web/local/venv/local/lib/python2.7/site-packages/pyswagger/primitives/_array.py", line 37, in apply_with self.extend(map(functools.partial(ctx['factory'].produce, obj.items), val)) File "/home/swen/style/web/local/venv/local/lib/python2.7/site-packages/pyswagger/primitives/init.py", line 186, in produce val = ret.apply_with(obj, val, ctx) File "/home/swen/style/web/local/venv/local/lib/python2.7/site-packages/pyswagger/primitives/_model.py", line 29, in apply_with self[k] = ctx['factory'].produce(pobj, v) File "/home/swen/style/web/local/venv/local/lib/python2.7/site-packages/pyswagger/primitives/init.py", line 180, in produce ret = creater(obj, val, ctx) File "/home/swen/style/web/local/venv/local/lib/python2.7/site-packages/pyswagger/primitives/_str.py", line 18, in create_str r = str(v) UnicodeEncodeError: 'ascii' codec can't encode characters in position 56-57: ordinal not in range(128)

mission-liao commented 8 years ago

The encoding problem is not handled very well so far, and "convert utf-8 byte stream back to str instance" is not covered in test case, I plan to fix it in this way for a temporary solution and add some test cases for it:

# instead of 
r = str(v)

# 'if everything is utf-8' case
if isnstance(v, six.binary_type):
    r = v.decode('utf-8')
else:
    r = str(v)

if something is unicode / utf-16 / ..., we need a long term solution, which is tracking in another issue.