Perl-Critic / PPI

54 stars 44 forks source link

Exception "Encountered unexpected character" is thrown in case of non-latin variable names. #226

Open kberov opened 6 years ago

kberov commented 6 years ago

I have a project in which I experiment with the ability of Perl to use unicode characters for naming variables and subroutines. I also use Test::Perl::Critic to maintain high quality of the code. After grepping from where the error comes I found that it is because PPI cannot parse such characters.

The following code fails with the following exception:

has domain => sub {
  my $дом = $_[0]->req->headers->host;
  $дом =~ s/(\:\d+)$//;    # remove port
  return $дом;
};
not ok - Test::Perl::Critic for "lib/Slovo/Controller.pm"
#   Failed test 'Test::Perl::Critic for "lib/Slovo/Controller.pm"'
#   at /home/berov/perl5/perlbrew/perls/perl-5.26.1/lib/site_perl/5.26.1/Test/Perl/Critic.pm line 121.
# 
# Perl::Critic had errors in "lib/Slovo/Controller.pm":
#   Can't parse code: Encountered unexpected character '208'

The exception is thrown from perlbrew/perls/perl-5.26.1/lib/site_perl/5.26.1/PPI/Token/Whitespace.pm:437 The Version of PPI is: 1.236

choroba commented 5 years ago

It breaks if you use non-ascii characters in unquoted LHS of the fat comma, too:

use utf8;
my %czech_to_english = ( kůň => 'horse' );