search

sergot / http-useragent

Web user agent class for Perl 6.
MIT License
36 stars 39 forks source link

Incorrect default encoding #207

Open zoffixznet opened 6 years ago

zoffixznet commented 6 years ago

The site http://eckva.net does not set encoding in its Content-type header. I believe this module default to an incorrect encoding (ASCII) in such cases, as that site works fine in browsers, Perl 5's version of the LWP::Simple module, and Perl 6's Cro::HTTP::Client:

perl -MLWP::Simple -wlE 'say get "http://eckva.net"'
`[...] <proper content> [...]
$ perl6 -MHTTP::UserAgent -e 'with HTTP::UserAgent.new { with .get("http://eckva.net") { } }'
Will not decode invalid ASCII (code point > 127 found)
  in method new at /home/zoffix/rakudo/install/share/perl6/site/sources/6D4B1B5FD49D9A74ACD442BFC9C3D3BE1E30A433 (HTTP::Response) line 23
  in method get-response at /home/zoffix/rakudo/install/share/perl6/site/sources/C030F6565540556B971FCFA11470E65E30846370 (HTTP::UserAgent) line 307
  in method request at /home/zoffix/rakudo/install/share/perl6/site/sources/C030F6565540556B971FCFA11470E65E30846370 (HTTP::UserAgent) line 159
  in method get at /home/zoffix/rakudo/install/share/perl6/site/sources/C030F6565540556B971FCFA11470E65E30846370 (HTTP::UserAgent) line 102
  in method get at /home/zoffix/rakudo/install/share/perl6/site/sources/C030F6565540556B971FCFA11470E65E30846370 (HTTP::UserAgent) line 105
  in block <unit> at -e line 1

$ perl6 -v
This is Rakudo version 2018.05-54-g148d7c5 built on MoarVM version 2018.05-57-g89dee3b
implementing Perl 6.c.

$ zef info HTTP::UserAgent
- Identity: HTTP::UserAgent:ver<1.1.44>:auth<github:sergot>