search

godfat / rest-core

Various rest-builder middleware for building REST clients.
Apache License 2.0
57 stars 13 forks source link

"\xAD" from ASCII-8BIT to UTF-8 ERROR when requested GET #27

Open AshwiniDoddamaniFluke opened 7 years ago

AshwiniDoddamaniFluke commented 7 years ago

`RestCore: spent 7.070209 Requested GET https://streaming.preprod.connect.fluke.com/iot-service/history/reports/download/b9498f13-62cf-44ba-9a1e-1eb2b4444d22?deviceType=fc3540&frequency=TWELVE_HOUR&sessionStartTime=1499763413506&from=1501668000000&reportType=xlsx&requesterTimeZone=GMT%2B0530&app_platform=website&app_vers=272

"\xAD" from ASCII-8BIT to UTF-8

/.rvm/gems/ruby-2.3.3/gems/activesupport-5.0.0/lib/active_support/core_ext/object/json.rb:34:in encode' /.rvm/gems/ruby-2.3.3/gems/activesupport-5.0.0/lib/active_support/core_ext/object/json.rb:34:into_json' /.rvm/gems/ruby-2.3.3/gems/activesupport-5.0.0/lib/active_support/core_ext/object/json.rb:34:in to_json' /.rvm/gems/ruby-2.3.3/gems/activesupport-5.0.0/lib/active_support/json/encoding.rb:55:into_json' /.rvm/gems/ruby-2.3.3/gems/json-1.8.6/lib/json/common.rb:223:in generate' /.rvm/gems/ruby-2.3.3/gems/json-1.8.6/lib/json/common.rb:223:ingenerate' /.rvm/gems/ruby-2.3.3/gems/activesupport-5.0.0/lib/active_support/json/encoding.rb:99:in stringify' /.rvm/gems/ruby-2.3.3/gems/activesupport-5.0.0/lib/active_support/json/encoding.rb:33:inencode' /.rvm/gems/ruby-2.3.3/gems/activesupport-5.0.0/lib/active_support/json/encoding.rb:20:in encode' /.rvm/gems/ruby-2.3.3/gems/activesupport-5.0.0/lib/active_support/core_ext/object/json.rb:37:into_json' /.rvm/gems/ruby-2.3.3/gems/actionpack-5.0.0/lib/action_controller/metal/renderers.rb:159:in block in <module:Renderers>' /.rvm/gems/ruby-2.3.3/gems/actionpack-5.0.0/lib/action_controller/metal/renderers.rb:152:inblock in _render_to_body_with_renderer' /.rvm/rubies/ruby-2.3.3/lib/ruby/2.3.0/set.rb:306:in each_key' /.rvm/rubies/ruby-2.3.3/lib/ruby/2.3.0/set.rb:306:ineach' /.rvm/gems/ruby-2.3.3/gems/actionpack-5.0.0/lib/action_controller/metal/renderers.rb:148:in _render_to_body_with_renderer' /.rvm/gems/ruby-2.3.3/gems/actionpack-5.0.0/lib/action_controller/metal/renderers.rb:144:inrender_to_body' /.rvm/gems/ruby-2.3.3/gems/actionpack-5.0.0/lib/abstract_controller/rendering.rb:26:in render' /.rvm/gems/ruby-2.3.3/gems/actionpack-5.0.0/lib/action_controller/metal/rendering.rb:36:inrender' /.rvm/gems/ruby-2.3.3/gems/actionpack-5.0.0/lib/action_controller/metal/instrumentation.rb:44:in block (2 levels) in render' /.rvm/gems/ruby-2.3.3/gems/activesupport-5.0.0/lib/active_support/core_ext/benchmark.rb:12:inblock in ms'`

godfat commented 7 years ago

Anymore information? Like the full backtrace, the original responses, etc.

AshwiniDoddamaniFluke commented 7 years ago

`Requested GET https: //streaming.preprod.connect.fluke.com/iot-service/history/reports/download/b9498f13-62cf-44ba-9a1e-1eb2b4444d22?deviceType=fc3540&frequency=TWELVE_HOUR&sessionStartTime=1499763413506&from=1502877600000&reportType=xlsx&requesterTimeZone=GMT%2B0530&app_platform=website&app_vers=270

RestBuilder::Promise: ERROR: invalid byte sequence in UTF - 8

from["/Users/ashwini/.rvm/gems/ruby-2.3.3/gems/rest-core-4.0.1/lib/rest-core/util/json.rb:75:in sub'", "/Users/ashwini/.rvm/gems/ruby-2.3.3/gems/rest-core-4.0.1/lib/rest-core/util/json.rb:75:instrip_bom'", "/Users/ashwini/.rvm/gems/ruby-2.3.3/gems/rest-core-4.0.1/lib/rest-core/util/json.rb:66:in normalize'", "/Users/ashwini/.rvm/gems/ruby-2.3.3/gems/rest-core-4.0.1/lib/rest-core/middleware/json_response.rb:34:inprocess'", "/Users/ashwini/.rvm/gems/ruby-2.3.3/gems/rest-core-4.0.1/lib/rest-core/middleware/json_response.rb:26:in block in call'", "/Users/ashwini/.rvm/gems/ruby-2.3.3/gems/rest-core-4.0.1/lib/rest-core/middleware/common_logger.rb:13:inblock in call'", "/Users/ashwini/.rvm/gems/ruby-2.3.3/gems/rest-core-4.0.1/lib/rest-core/middleware/cache.rb:34:in block in app_call'", "/Users/ashwini/.rvm/gems/ruby-2.3.3/gems/promise_pool-0.9.0/lib/promise_pool/promise.rb:118:inblock in resolve'", "/Users/ashwini/.rvm/gems/ruby-2.3.3/gems/promise_pool-0.9.0/lib/promise_pool/promise.rb:118:in each'", "/Users/ashwini/.rvm/gems/ruby-2.3.3/gems/promise_pool-0.9.0/lib/promise_pool/promise.rb:118:ininject'", "/Users/ashwini/.rvm/gems/ruby-2.3.3/gems/promise_pool-0.9.0/lib/promise_pool/promise.rb:118:in resolve'", "/Users/ashwini/.rvm/gems/ruby-2.3.3/gems/promise_pool-0.9.0/lib/promise_pool/promise.rb:109:infulfilling'", "/Users/ashwini/.rvm/gems/ruby-2.3.3/gems/promise_pool-0.9.0/lib/promise_pool/promise.rb:84:in block in fulfill'", "/Users/ashwini/.rvm/gems/ruby-2.3.3/gems/promise_pool-0.9.0/lib/promise_pool/promise.rb:84:insynchronize'", "/Users/ashwini/.rvm/gems/ruby-2.3.3/gems/promise_pool-0.9.0/lib/promise_pool/promise.rb:84:in fulfill'", "/Users/ashwini/.rvm/gems/ruby-2.3.3/gems/promise_pool-0.9.0/lib/promise_pool/promise.rb:136:inprotected_yield'", "/Users/ashwini/.rvm/gems/ruby-2.3.3/gems/promise_pool-0.9.0/lib/promise_pool/promise.rb:49:in block in defer'", "/Users/ashwini/.rvm/gems/ruby-2.3.3/gems/rest-builder-0.9.1/lib/rest-builder/engine.rb:28:incall'", "/Users/ashwini/.rvm/gems/ruby-2.3.3/gems/rest-core-4.0.1/lib/rest-core/middleware/auth_basic.rb:10:in call'", "/Users/ashwini/no-rocket-science/sw-web/lib/fluke/token.rb:24:incall'", "/Users/ashwini/.rvm/gems/ruby-2.3.3/gems/rest-core-4.0.1/lib/rest-core/middleware/cache.rb:29:in app_call'", "/Users/ashwini/.rvm/gems/ruby-2.3.3/gems/rest-core-4.0.1/lib/rest-core/middleware/cache.rb:25:incall'", "/Users/ashwini/.rvm/gems/ruby-2.3.3/gems/rest-core-4.0.1/lib/rest-core/middleware/common_logger.rb:12:in call'",

godfat commented 7 years ago

This means that the response was claiming to be UTF-8 encoded however it's not. Do you have the original response available?

Suppose this is really the case, what could we do here? We can't reliably parse the JSON without knowing the correct encoding. We could somehow try to fix the response, but that would cause data loss and might not give desired result.

SunnyKukkarFluke commented 7 years ago

Hi @godfat,

We are getting an xlsx file as response of this API. Here are the response headers which we are getting in postman:

connection →keep-alive content-disposition →attachment; filename=FC3540Report1503054716877.xlsx content-length →189062 content-type →application/xlsx date →Fri, 18 Aug 2017 11:11:57 GMT expires →: 0 x-application-context →application:7300

godfat commented 7 years ago

We need the response body to figure out the encoding issue. However on the other hand, it seems like the response was actually an XLSX file, which is not JSON, therefore you shouldn't use JsonResponse middleware.

What's the actually code you're using? If you're using Universal, you should disable :json_response.

SunnyKukkarFluke commented 7 years ago

godfat,

Once we are calling get api of rest core using "rest_core_obj.request_full(data)" to download XLSX. I am also surprised even if in response header the content-type is mentioned as XLSX, why its encoding as json.

godfat commented 7 years ago

@SunnyKukkarFluke You could try:

rest_core_obj.request_full(data.merge(:json_response => false))

The built-in middleware was not designed to download anything, so if you want to do different stuffs with the same client object, then you should write your own middleware.