p5pRT
commented
7 years ago From rafal@zorro.ztk-rp.eu
Created by rafal@zorro.ztk-rp.eu
After upgrading from debian-wheezy to debian-jessie HTML::Mason started to behave strangely with respect to UTF8 encoding. Earlier both web-pages and forms were working correctly (in UTF8) without any special setup. As of jessie with Apache 2.4 UTF8 no longer works. 1. I had to add binmode(STDOUT\,'UTF8') to modules. 2. I had to decode_utf8($_) data from forms before passing them over to psql-db This report I file with example code of erratic behavior of Text::CSV::Encoded since I could narrow the problem to just a few lines of test-case.
======================== #!/usr/bin/perl use Text::CSV::Encoded; open(my $FH\, shift) or die "open"; binmode($FH\, ":encoding(cp1250) :raw :bytes"); local $/ = "\r\n"; my $csv = Text::CSV::Encoded->new ( { encoding_in => "cp1250"\, binary => 1\, eol => $/\, sep_char => ';'\, } ) or die "Cannot use CSV: ".Text::CSV->error_diag (); $\ = "\n"; while ( \<$FH> ) { s/\s+$//; print; if ($csv->parse( $_ )) { print $csv->fields(); } } __END__ 10;"SPӣDZIELNIA WARSZAWA";62;"TEST"
In this example: 1. the test file (provided "inline") as \ contains two speciffic characters from CODE-PAGE-1250\, one such char just after another. 1a. this test file IS-NOT UTF8 encoded. 2. the input stream is correctly marked as CP1250 3. the module gets correct information as to that file encoding ... and yet\, the module complains about encoutering a "wide-char"\, which in the above setup should not ever be possible (I think).
The result of the above program is:
$ ./wide-char test-input 10;"SPӣDZIELNIA WARSZAWA";62;"TEST" Wide character in subroutine entry at /usr/share/perl5/Text/CSV/Encoded/Coder/Encode.pm line 37\, \<$FH> chunk 1. $
This result is incorrect\, since the file does not contain any "wide chars".
Migrated from rt.perl.org#130199 (status was 'rejected')
Searchable as RT130199$