httprb / http

HTTP (The Gem! a.k.a. http.rb) - a fast Ruby HTTP client with a chainable API, streaming support, and timeouts
MIT License
3.01k stars 321 forks source link

strange response body #708

Closed ronald closed 2 years ago

ronald commented 2 years ago

Sorry, can someone please help me with this unexpected response body?

result

HTTP.get("https://tpl.mmcdn.de/min/?g=1047_pc5847").body.to_s
=> "\u001F\x8B\b\u0000\u0000\u0000\u0000\u0000\u0002\u0003m\x90Y\n\xC3 \u0018\x84\u000F\x93g\xC1,f\xEBi4\xFE&\x92\xC5b\fm)\xBD{\x8D\xA2\xD8\xE5u\xBE\x81Yz\xAD\x94y\"\xB4\u001At\xD5r\xA5\xA0\x91\xA0\x9AA\xDEg0\xB0\xAA\xA1\x97_V\xF4\x99\xA89e\xFC\u000F+-\xEB\xA0\xE6\xA5g;\xCC\xC7\xC6#\xED\xB3\u0012\xD7m\xC3<\x9C\xD5f4\xDDM`\u0003!\xACj=3p7\xB1\tƹ\xC0\xF8\e\u0014' E\u0000\xD3\u0018\xFC\xB7I\u001A\xF8\u0014\xAD\u0017:\xDE1\xE2e\u000E\x82\u001EK\b\u000E~7D?\x92\x92;\f\xCA\xD6?\xB5\xF8\x86܄J\u000EX\xE48\x99d\xB4u\xCF\xC9\u0014W\xD7{B\x8C\x93bV\xDC\xE6\xE4$\xD0o{\xBD\u0001b\xABQ\xB4\xA1\u0001\u0000\u0000"

expected result

uri = URI("https://tpl.mmcdn.de/min/?g=1047_pc5847")
Net::HTTP.get(uri) 
=> ":root{--mt-primaer-farbe1:#ecb47a;--mt-primaer-farbe2:#f6dabd;--mt-primaer-farbe3:#f9e6d3;--mt-sekundaer-farbe:#30687b;--mt-kontrast-farbe:#c55b48;--mt-text-farbe1:#001f00;--mt-text-farbe2:#005200;--mt-hg-farbe1:white;--mt-hg-farbe2:#e9d9b5;--mt-default-farbe:white;--primary:#30687b;--secondary:#ecb47a;--info:#f6dabd;--light:#f9e6d3;--dark:#c55b48;--text-light:white;--text-primary:#001f00;--text-secondary:#005200}"

context

HTTP::VERSION
=> "5.0.4"
tycooon commented 2 years ago

It looks like the server is returning gzipped vesion was some reason (you can check that using curl). Not sure how the net/http version is working, though.

ronald commented 2 years ago

Yes,

 HTTP.get("https://tpl.mmcdn.de/min/?g=1047_pc5847").headers['content-encoding']
=> "gzip"

But doesn't http do the ungzipping?

tycooon commented 2 years ago

It looks like you should use auto_inflate extension https://github.com/httprb/http/wiki/Compression.

ronald commented 2 years ago

That works. Thanks a lot.


☮️ Нет войне! ☮️

ronald commented 2 years ago

Sorry, problem reappeared.

Any other hints?

 HTTP.use(:auto_inflate).get("https://tpl.mmcdn.de/min/?g=1047_pc5847").body.to_s
=> "\u001F\x8B\b\u0000\u0000\u0000\u0000\u0000\u0002\u0003m\x90Y\n\xC3 \u0018\x84\u000F\x93g\xC1,f\xEBi4\xFE&\x92\xC5b\fm)\xBD{\x8D\xA2\xD8\xE5u\xBE\x81Yz\xAD\x94y\"\xB4\u001At\xD5r\xA5\xA0\x91\xA0\x9AA\xDEg0\xB0\xAA\xA1\x97_V\xF4\x99\xA89e\xFC\u000F+-\xEB\xA0\xE6\xA5g;\xCC\xC7\xC6#\xED\xB3\u0012\xD7m\xC3<\x9C\xD5f4\xDDM`\u0003!\xACj=3p7\xB1\tƹ\xC0\xF8\e\u0014' E\u0000\xD3\u0018\xFC\xB7I\u001A\xF8\u0014\xAD\u0017:\xDE1\xE2e\u000E\x82\u001EK\b\u000E~7D?\x92\x92;\f\xCA\xD6?\xB5\xF8\x86܄J\u000EX\xE48\x99d\xB4u\xCF\xC9\u0014W\xD7{B\x8C\x93bV\xDC\xE6\xE4$\xD0o{\xBD\u0001b\xABQ\xB4\xA1\u0001\u0000\u0000"
tycooon commented 2 years ago

Works for me:

[2] pry(main)> HTTP.use(:auto_inflate).get("https://tpl.mmcdn.de/min/?g=1047_pc5847").body.to_s
=> ":root{--mt-primaer-farbe1:#ecb47a;--mt-primaer-farbe2:#f6dabd;--mt-primaer-farbe3:#f9e6d3;--mt-sekundaer-farbe:#30687b;--mt-kontrast-farbe:#c55b48;--mt-text-farbe1:#001f00;--mt-text-farbe2:#005200;--mt-hg-farbe1:white;--mt-hg-farbe2:#e9d9b5;--mt-default-farbe:white;--primary:#30687b;--secondary:#ecb47a;--info:#f6dabd;--light:#f9e6d3;--dark:#c55b48;--text-light:white;--text-primary:#001f00;--text-secondary:#005200}"
ronald commented 2 years ago

I checked different projects:

For the older project running ruby 2.3 / http 4, it's currently not possible to upgrade to http 5 (needs ruby 2.5). So I will switch to faraday.

Since I can not reproduce the problem with the current version of http (5.0.4), I'll close the ticket.