search

sergot / http-useragent

Web user agent class for Perl 6.
MIT License
36 stars 39 forks source link

Dies when parsing requests with non-ASCII header values #236

Closed jjatria closed 3 years ago

jjatria commented 3 years ago

HTTP::UserAgent dies when processing responses that have non-ASCII header values:

$ raku -I. -MHTTP::UserAgent -e 'HTTP::UserAgent.new.get: "http://icanhazip.com"'
Will not decode invalid ASCII (code point (-16) < 0 found)
  in method new at /home/user/HTTP-UserAgent/lib/HTTP/Response.pm6 (HTTP::Response) line 23
  in method get-response at /home/user/HTTP-UserAgent/lib/HTTP/UserAgent.pm6 (HTTP::UserAgent) line 300
  in method request at /home/user/HTTP-UserAgent/lib/HTTP/UserAgent.pm6 (HTTP::UserAgent) line 159
  in method get at /home/user/HTTP-UserAgent/lib/HTTP/UserAgent.pm6 (HTTP::UserAgent) line 102
  in method get at /home/user/HTTP-UserAgent/lib/HTTP/UserAgent.pm6 (HTTP::UserAgent) line 106
  in block <unit> at -e line 1

For comparison:

Curl:

$ curl -v http://icanhazip.com
... [redacted]
< x-duck: 🦆
... [redacted]

Cro::HTTP

$ raku -MCro::HTTP::Client -e 'my $res = await Cro::HTTP::Client.new.get: "http://icanhazip.com"; say $res.headers.tail'
Cro::HTTP::Header.new(name => "x-duck", value => "ð\x[9F]¦\x[86]")

HTTP::Tiny

$ raku -MHTTP::Tiny -e 'dd HTTP::Tiny.get("http://icanhazip.com")<headers><x-duck>'
Str %headers = "ð\x[9F]¦\x[86]"