search

raku-community-modules / LWP-Simple

LWP::Simple quick & dirty implementation for Rakudo
http://www.streppone.it/cosimo/blog/tag/LWP::Simple/
Artistic License 2.0
8 stars 9 forks source link

Incorrect default encoding #29

Closed zoffixznet closed 5 years ago

zoffixznet commented 6 years ago

The site http://eckva.net does not set encoding in its Content-type header. I believe this module default to an incorrect encoding (ASCII) in such cases, as that site works fine in browsers, Perl 5's version of the module, and Perl 6's Cro::HTTP::Client:

perl -MLWP::Simple -wlE 'say get "http://eckva.net"'
`[...] <proper content> [...]
$ perl6 -MLWP::Simple -e 'say LWP::Simple.get: "http://eckva.net"'
Will not decode invalid ASCII (code point > 127 found)
  in method parse_response at /home/zoffix/rakudo/install/share/perl6/site/sources/5144DC413A3D6FEA44A2A1547406118DCF6B244D (LWP::Simple) line 328
  in method make_request at /home/zoffix/rakudo/install/share/perl6/site/sources/5144DC413A3D6FEA44A2A1547406118DCF6B244D (LWP::Simple) line 262
  in method request_shell at /home/zoffix/rakudo/install/share/perl6/site/sources/5144DC413A3D6FEA44A2A1547406118DCF6B244D (LWP::Simple) line 84
  in method get at /home/zoffix/rakudo/install/share/perl6/site/sources/5144DC413A3D6FEA44A2A1547406118DCF6B244D (LWP::Simple) line 31
  in block <unit> at -e line 1

$ perl6 -v
This is Rakudo version 2018.05-54-g148d7c5 built on MoarVM version 2018.05-57-g89dee3b
implementing Perl 6.c.

$ zef info LWP::Simple
- Identity: LWP::Simple:ver<0.103>:auth<Cosimo Streppone>
jonathanstowe commented 5 years ago

Changing it to latin-1 seems to work and it all passes all the tests. Though I'm not actually seeing where the non-ASCII character is coming from with that URL.

I'll go with that fix and see how it goes.