search

nahi / httpclient

'httpclient' gives something like the functionality of libwww-perl (LWP) in Ruby.
https://github.com/nahi/httpclient
701 stars 290 forks source link

How to handle content with encoding UTF-16 with BOM present? #413

Open PikachuEXE opened 4 years ago

PikachuEXE commented 4 years ago

I am using this library to download remote text files Some of them contains content with UTF-16LE (instead of usual UTF-8) with BOM (\xff\xfe) And directly handling the downloaded content would causes error reported in https://bugs.ruby-lang.org/issues/16402

Is there a way in this library to handle the encoding/BOM issue properly? (Without resorting to write to temp file and read again with different mode)