serokell / haskell-with-utf8

Get your IO right on the first try
https://serokell.io/blog/haskell-with-utf8
52 stars 3 forks source link

Use utf-8 codepage on Windows #16

Open Sorokin-Anton opened 1 year ago

Sorokin-Anton commented 1 year ago

Currently withUtf8 is not adding full utf-8 support on Windows, because we're not changing active codepage, and default one is not supporting utf-8 symbols.

Steps to reprouce:

Create a.hs with

import Main.Utf8
import System.IO.CodePage

main = do
    withUtf8 $ putStrLn "\9920"
    withUtf8 $ withCP65001 $ putStrLn "\9920"
    withCP65001 $ putStrLn "\9920"
    putStrLn "\9920"

and run it in cmd (my machine has Windows 10 installed). Output is

?
⛀
⛀
a.hs: <stdout>: commitBuffer: invalid argument (invalid character)

so when default codepage is used, symbols are rendered as question marks. When correct codepage is set, printing works with or without withUtf8 and when neither withCP65001 nor withUtf8 was used, there is error.

Suggestion

We can include withCP65001 to withUtf8. It makes printing user-friendly on Windows and does nothing on other systems.