ql-io / ql.io

A node.js based declarative, data-retrieval and aggregation gateway for quickly consuming HTTP APIs
http://ql.io
Other
931 stars 112 forks source link

Get EINVAL for valid utf-8 payload #548

Closed shimonchayim closed 12 years ago

shimonchayim commented 12 years ago

The issue seems to because we are doing Iconv conversion on chunks. Likely there is a byte alignment issues. Following code in verb.js seems to be the problem.

this['parse response'] = function(args) { // Just append bufs to a string var encoding = 'UTF-8'; if(args.headers['content-type']) { var contentType = headers.parse('content-type', args.headers['content-type'] || ''); encoding = contentType.params.charset ? contentType.params.charset : contentType.subtype === 'csv' ? 'ASCII' : 'UTF-8'; } var str = ''; var iconv = new Iconv(encoding, 'UTF-8'); _.each(args.body, function(buf) { buf = (iconv.convert(buf)); str += buf.toString('UTF-8'); }); return { content: str }; };

shimonchayim commented 12 years ago

Can you also port the changes to 0.6 Branch?

idralyuk commented 12 years ago

Done: https://github.com/ql-io/ql.io/commit/5641082254688eb9aae018a20689b0dbc87739b7