source of template string determines output encoding under `use utf8;`

[wchristian]

I have uploaded a minimal test case for this issue here:

http://dl.dropbox.com/u/10190786/encoding.zip

What happens for me is that test_result_works is written as a file with proper unicode content; but writing test_result_broken causes the script to die with an error about wide characters.

The difference between those is that the working file is written with a template defined within the perl code itself; while the broken file template comes from the DATA section. Earlier testing indicates that in the first case, Text::Xslate returns bytes and in the second case character. use utf8 is in effect for both of those, so the difference in behavior is unexpected.

[punytan]

How about this verbose version? You should see FLAGS carefully. See also perlunifaq

use strictures;

package CPANRSS;

use utf8;
use File::Slurp qw' write_file read_file ';
use Data::Section::Simple 'get_data_section';
use Text::Xslate;
use Devel::Peek;
use Encode;
$| = 1;

save_feed();

sub save_feed {
    my $name = read_file 'test_utf8';
    show("name", $name);
    $name = decode_utf8 $name;
    show("name", $name);

    my $works = Text::Xslate->new->render_string( 'wagh: <: $name :>', { name => $name } );
    show("works", $works);

    $works = encode_utf8($works);
    show("works", $works);

    write_file 'test_result_works', { binmode => ':raw' }, $works;

    my $template = get_data_section( 'meep' );
    show("template", $template);

    my $broken = Text::Xslate->new->render_string( $template, { name => $name } );
    show("broken", $broken);

    $broken = encode_utf8($broken);
    show("broken", $broken);

    write_file 'test_result_broken', { binmode => ':raw' }, $broken;
}

sub show {
    print "\$$_[0]: ";
    Dump $_[1];
}

1;

__DATA__

@@ meep
wagh: <: $name :>