UTF-8 problem in scanner

                                                                                    [          ]

Issue migrated (2015-06-05) from old issue tracker http://mirrorbrain.org/issues/issue149

Title    UTF-8 problem in scanner
 Priority   bug    Status   resolved
Superseder        Nosy List poeml
Assigned To poeml Keywords  scanner

msg538 (view) Author: poeml Date: 2014-02-09.13:57:24

Cool... I found a fix here but can't remember why I made it. Now I found the bug and confirmed that the scanner is indeed broken when it encounters files with UTF-8 characters:

Character in 'c' format wrapped in pack at tools/scanner.pl line 597. Character in 'c' format wrapped in pack at tools/scanner.pl line 597. Character in 'c' format wrapped in pack at tools/scanner.pl line 597. Character in 'c' format wrapped in pack at tools/scanner.pl line 597. Character in 'c' format wrapped in pack at tools/scanner.pl line 597. Character in 'c' format wrapped in pack at tools/scanner.pl line 597. Character in 'c' format wrapped in pack at tools/scanner.pl line 597. Character in 'c' format wrapped in pack at tools/scanner.pl line 597. Character in 'c' format wrapped in pack at tools/scanner.pl line 597. Character in 'c' format wrapped in pack at tools/scanner.pl line 597. Character in 'c' format wrapped in pack at tools/scanner.pl line 597. Character in 'c' format wrapped in pack at tools/scanner.pl line 597.

msg539 (view) Author: poeml Date: 2014-02-09.13:59:14

The following makes it work. Committed in r8392.

@@ -594,7 +594,7 @@ print "$identifier: date $date\n"; print "$identifier: size $size\n"; }

$name1 =~ s{%([\da-fA-F]{2})}{pack 'c', hex $1}ge;
$name1 =~ s{%([\da-fA-F]{2})}{pack 'U', hex $1}ge; $name1 =~ s{^./}{}; my $dir = 1 if $pre =~ m{"[DIR]"};
print "$identifier: $pre^$name1^$date^$size\n" if $verbose > 1;

@@ -651,7 +651,7 @@ print "$identifier: date $date\n"; print "$identifier: size $size\n"; }
$name1 =~ s{%([\da-fA-F]{2})}{pack 'c', hex $1}ge;
$name1 =~ s{%([\da-fA-F]{2})}{pack 'U', hex $1}ge; $name1 =~ s{^./}{}; my $dir = 1 if $pre =~ m{>Directory<}; my $t = length($name) ? "$name/$name1" : $name1;

History
         Date         User  Action              Args
2014-02-09 13:59:14 poeml set    status: in-progress -> resolved
                                   messages: + msg539
2014-02-09 13:57:24 poeml create

poeml / mirrorbrain