search

tokiwa-software / fuzion

The Fuzion Language Implementation
https://fuzion-lang.dev
GNU General Public License v3.0
47 stars 11 forks source link

windows: output of utf-8 not working #2586

Open michaellilltokiwa opened 9 months ago

michaellilltokiwa commented 9 months ago

On my machine chcp.com 65001 does the trick. Not sure why this does not apply to github...

sam@DESKTOP-CA0927S UCRT64 ~/fuzion
# ./build/bin/fz -modules=java.base tests/javaBase/javaHello.fz
UTF-8
Hello Java ?!
Hello Java ?!
Hello Java ?!
Hello Java ?!
Hello Java ?!
string has 16 bytes: [72, 101, 108, 108, 111, 32, 74, 97, 118, 97, 32, -16, -97, -116, -115, 33]
Hello Java ­ƒîì!
[1, 2, 3]
[instance[Java.java.io.PrintStream]]
[false, true, false, true]
[0, 1, -128, 127]
[0, 1, -32768, 32767]
[0, 1, 0, 65535]
[0, 1, -2147483648, 2147483647]
[0, 1, -9223372036854775808, 9223372036854775807]
[0.0, 1.0, 3.1415927, 3.4028235E38]
[0.0, 1.0, 3.141592653589793, 1.7976931348623157E308]

sam@DESKTOP-CA0927S UCRT64 ~/fuzion
# chcp.com 65001
Aktive Codepage: 65001.

sam@DESKTOP-CA0927S UCRT64 ~/fuzion
# ./build/bin/fz -modules=java.base tests/javaBase/javaHello.fz 
UTF-8
Hello Java 🌍!
Hello Java 🌍!
Hello Java 🌍!
Hello Java 🌍!
Hello Java 🌍!
string has 16 bytes: [72, 101, 108, 108, 111, 32, 74, 97, 118, 97, 32, -16, -97, -116, -115, 33]
Hello Java 🌍!
[1, 2, 3]
[instance[Java.java.io.PrintStream]]
[false, true, false, true]
[0, 1, -128, 127]
[0, 1, -32768, 32767]
[0, 1, 0, 65535]
[0, 1, -2147483648, 2147483647]
[0, 1, -9223372036854775808, 9223372036854775807]
[0.0, 1.0, 3.1415927, 3.4028235E38]
[0.0, 1.0, 3.141592653589793, 1.7976931348623157E308]

sam@DESKTOP-CA0927S UCRT64 ~/fuzion
#
michaellilltokiwa commented 9 months ago

Problem is not just on github but also my machine. Somehow the world-emoji gets garbled when piping to file.

@maxteufel @fridis Do you have any ideas why this might be happening? I think this is Javas fault? and not the fault of the pipe/terminal?

# ./build/bin/fz -modules=java.base tests/javaBase/javaHello.fz 
Aktive Codepage: 65001.
UTF-8
Hello Java 🌍!
Hello Java 🌍!
Hello Java 🌍!
Hello Java 🌍!
Hello Java 🌍!
string has 16 bytes: [72, 101, 108, 108, 111, 32, 74, 97, 118, 97, 32, -16, -97, -116, -115, 33]
Hello Java 🌍!
[1, 2, 3]
[instance[Java.java.io.PrintStream]]
[false, true, false, true]
[0, 1, -128, 127]
[0, 1, -32768, 32767]
[0, 1, 0, 65535]
[0, 1, -2147483648, 2147483647]
[0, 1, -9223372036854775808, 9223372036854775807]
[0.0, 1.0, 3.1415927, 3.4028235E38]
[0.0, 1.0, 3.141592653589793, 1.7976931348623157E308]

sam@DESKTOP-CA0927S UCRT64 ~/fuzion
# ./build/bin/fz -modules=java.base tests/javaBase/javaHello.fz > asdf

sam@DESKTOP-CA0927S UCRT64 ~/fuzion
# cat asdf
Aktive Codepage: 65001.
UTF-8
Hello Java ?!
Hello Java ?!
Hello Java ?!
Hello Java ?!
Hello Java ?!
string has 16 bytes: [72, 101, 108, 108, 111, 32, 74, 97, 118, 97, 32, -16, -97, -116, -115, 33]
Hello Java 🌍!
[1, 2, 3]
[instance[Java.java.io.PrintStream]]
[false, true, false, true]
[0, 1, -128, 127]
[0, 1, -32768, 32767]
[0, 1, 0, 65535]
[0, 1, -2147483648, 2147483647]
[0, 1, -9223372036854775808, 9223372036854775807]
[0.0, 1.0, 3.1415927, 3.4028235E38]
[0.0, 1.0, 3.141592653589793, 1.7976931348623157E308]
maxteufel commented 9 months ago

I personally would rather believe this is the fault of the shell, which might not be piping correctly. If it was Java's fault why should it work when not piping. To confirm this, you could probably create a UTF-8 file with the world emoji, and then cat and pipe.

michaellilltokiwa commented 9 months ago

@maxteufel catting and pipeing seems to work just fine:

sam@DESKTOP-CA0927S UCRT64 ~
# cat fuzion/build/tests/javaBase/javaHello.fz.expected_out 
Hello Java 🌍!
Hello Java 🌍!
Hello Java 🌍!
Hello Java 🌍!
Hello Java 🌍!
string has 16 bytes: [72, 101, 108, 108, 111, 32, 74, 97, 118, 97, 32, -16, -97, -116, -115, 33]
Hello Java 🌍!
[1, 2, 3]
[instance[Java.java.io.PrintStream]]
[false, true, false, true]
[0, 1, -128, 127]
[0, 1, -32768, 32767]
[0, 1, 0, 65535]
[0, 1, -2147483648, 2147483647]
[0, 1, -9223372036854775808, 9223372036854775807]
[0.0, 1.0, 3.1415927, 3.4028235E38]
[0.0, 1.0, 3.141592653589793, 1.7976931348623157E308]

sam@DESKTOP-CA0927S UCRT64 ~
# cat fuzion/build/tests/javaBase/javaHello.fz.expected_out > hello

sam@DESKTOP-CA0927S UCRT64 ~
# cat hello
Hello Java 🌍!
Hello Java 🌍!
Hello Java 🌍!
Hello Java 🌍!
Hello Java 🌍!
string has 16 bytes: [72, 101, 108, 108, 111, 32, 74, 97, 118, 97, 32, -16, -97, -116, -115, 33]
Hello Java 🌍!
[1, 2, 3]
[instance[Java.java.io.PrintStream]]
[false, true, false, true]
[0, 1, -128, 127]
[0, 1, -32768, 32767]
[0, 1, 0, 65535]
[0, 1, -2147483648, 2147483647]
[0, 1, -9223372036854775808, 9223372036854775807]
[0.0, 1.0, 3.1415927, 3.4028235E38]
[0.0, 1.0, 3.141592653589793, 1.7976931348623157E308]

sam@DESKTOP-CA0927S UCRT64 ~
maxteufel commented 9 months ago

is it this issue perhaps: https://stackoverflow.com/a/11930722

michaellilltokiwa commented 9 months ago

@maxteufel Could very well be, yes.