search

richterger / Perl-LanguageServer

Language Server for Perl
Other
219 stars 53 forks source link

enable utf8 on stdout and stderr #167

Closed wielandp closed 1 year ago

wielandp commented 1 year ago

Data is always utf8 on stdout and stderr.

wielandp commented 1 year ago

should fix issue with cyrillic output https://github.com/richterger/Perl-LanguageServer/issues/76

wielandp commented 1 year ago

Testing UTF8

cat test_utf8_01.pl

use strict;
use warnings;

#use utf8;

# binmode(STDOUT, ":utf8");
# binmode(STDERR, ":utf8");

print "Здравейте folks\n";

print STDERR "Error Здравейте at ".__FILE__.":".__LINE__."\n";

die "asdf Здравейте";

cat test_utf8_02.pl

use strict;
use warnings;

use utf8;

# binmode(STDOUT, ":utf8");
# binmode(STDERR, ":utf8");

print "Здравейте folks\n";

print STDERR "Error Здравейте at ".__FILE__.":".__LINE__."\n";

die "asdf Здравейте";

cat test_utf8_03.pl

use strict;
use warnings;

#use utf8;

binmode(STDOUT, ":utf8");
binmode(STDERR, ":utf8");

print "Здравейте folks\n";

print STDERR "Error Здравейте at ".__FILE__.":".__LINE__."\n";

die "asdf Здравейте";

cat test_utf8_04.pl

use strict;
use warnings;

use utf8;

binmode(STDOUT, ":utf8");
binmode(STDERR, ":utf8");

print "Здравейте folks\n";

print STDERR "Error Здравейте at ".__FILE__.":".__LINE__."\n";

die "asdf Здравейте";   
wp@DESKTOP-5OCF6FQ:~/src/pls$

Terminal in vscode

wp@DESKTOP-5OCF6FQ:~/src/pls$ perl test_utf8_01.pl 
Здравейте folks
Error Здравейте at test_utf8_01.pl:11
asdf Здравейте at test_utf8_01.pl line 13.
wp@DESKTOP-5OCF6FQ:~/src/pls$ perl test_utf8_02.pl 
Wide character in print at test_utf8_02.pl line 9.
Здравейте folks
Wide character in print at test_utf8_02.pl line 11.
Error Здравейте at test_utf8_02.pl:11
Wide character in die at test_utf8_02.pl line 13.
asdf Здравейте at test_utf8_02.pl line 13.
wp@DESKTOP-5OCF6FQ:~/src/pls$ perl test_utf8_03.pl 
ÐдÑавейÑе folks
Error ÐдÑавейÑе at test_utf8_03.pl:11
asdf ÐдÑавейÑе at test_utf8_03.pl line 13.
wp@DESKTOP-5OCF6FQ:~/src/pls$ perl test_utf8_04.pl 
Здравейте folks
Error Здравейте at test_utf8_04.pl:11
asdf Здравейте at test_utf8_04.pl line 13.

Debug console test_utf8_01.pl: same output but different order

Error Здравейте at /home/wp/src/pls/test_utf8_01.pl:11
asdf Здравейте at /home/wp/src/pls/test_utf8_01.pl line 13.
Здравейте folks

Debug console test_utf8_02.pl: same output

Wide character in print at /home/wp/src/pls/test_utf8_02.pl line 9.
Здравейте folks
Wide character in print at /home/wp/src/pls/test_utf8_02.pl line 11.
Error Здравейте at /home/wp/src/pls/test_utf8_02.pl:11
Wide character in die at /home/wp/src/pls/test_utf8_02.pl line 13.
asdf Здравейте at /home/wp/src/pls/test_utf8_02.pl line 13.

Debug console test_utf8_03.pl: same output

Здравейте folks
Error Здравейте at /home/wp/src/pls/test_utf8_03.pl:11
asdf Здравейте at /home/wp/src/pls/test_utf8_03.pl line 13.

Debug console test_utf8_04.pl: same output

Здравейте folks
Error Здравейте at /home/wp/src/pls/test_utf8_04.pl:11
asdf Здравейте at /home/wp/src/pls/test_utf8_04.pl line 13.
wielandp commented 1 year ago

It would be possible to do the decode earlier (after the read), but I did it just before the problem starts.

richterger commented 1 year ago

This only work if your program uses utf8. Any other locale will not work. Also most programms nowadays use utf8, it might be a good idea to make this more generic. I think https://metacpan.org/pod/Encode::Locale should do the job.

Could you rewrite your pull request to use https://metacpan.org/pod/Encode::Locale?