when the file with Char out of ASCII, mostly will code 1 exit

[soiamsoNG]

main :: IO ()
main = do
  let z = "输入法"
  putStrLn z

[Enter steps to reproduce below:]

1. when the file with Char out of ASCII, like above code, move the arrow key or save
2. try ghc-mod in command line under Windows cmd no Exception.

Atom Version: 1.5.0-beta3 System: Unknown Windows Version Thrown From: haskell-ghc-mod package, v1.6.4

Stack Trace

Haskell-ghc-mod: ghc-mod interactive command lint failed with error code 1

At caps: {"version":[5,5,0,0],"legacyInteractive":true,"fileMap":true,"rootExec":false,"quoteArgs":true}
URI: C:\Users\Coder\Desktop\test.hs
Args: 
message: ghc-mod: <stdin>: hGetLine: invalid argument (invalid byte sequence)

log:
-2.156s: getProcessOptions(C:\Users\Coder\Desktop)
-2.08s: PATH = C:\ProgramData\Oracle\Java\javapath;C:\Program Files\Haskell\bin;C:\Program Files\Haskell Platform\7.10.3\lib\extralibs\bin;C:\Program Files\Haskell Platform\7.10.3\bin;C:\WINDOWS\system32;C:\WINDOWS;C:\WINDOWS\System32\Wbem;C:\WINDOWS\System32\WindowsPowerShell\v1.0\;C:\Program Files\Haskell Platform\7.10.3\mingw\bin;C:\ProgramData\chocolatey\bin;C:\Users\Coder\AppData\Local\atom\bin
-1.377s: warn: nameless module in C:\Users\Coder\Desktop\test.hs
-1.374s: Prelude created
-1.373s: Prelude updating
-1.274s: Prelude moduleName mismatch: undefined != Prelude
-1.188s: Prelude moduleName mismatch: undefined != Prelude
-1.102s: Prelude moduleName mismatch: undefined != Prelude
-0.962s: {"version":[5,5,0,0],"legacyInteractive":true,"fileMap":true,"rootExec":false,"quoteArgs":true}
-0.956s: running C:\Users\Coder\AppData\Roaming\cabal\bin\ghc-mod lang with options.cwd = C:\Users\Coder\Desktop,options.env = [object Object],options.encoding = utf-8
-0.945s: running C:\Users\Coder\AppData\Roaming\cabal\bin\ghc-mod flag with options.cwd = C:\Users\Coder\Desktop,options.env = [object Object],options.encoding = utf-8
-0.893s: running C:\Users\Coder\AppData\Roaming\cabal\bin\ghc-mod browse,-d,Prelude with options.cwd = C:\Users\Coder\Desktop,options.env = [object Object],options.encoding = utf-8
-0.585s: Trying to run ghc-modi in C:\Users\Coder\Desktop
-0.584s: Failed. Falling back to ghc-mod
-0.583s: running C:\Users\Coder\AppData\Roaming\cabal\bin\ghc-mod --map-file,C:\Users\Coder\Desktop\test.hs,lint,C:\Users\Coder\Desktop\test.hs with options.cwd = C:\Users\Coder\Desktop,options.env = [object Object],options.encoding = utf-8
-0.569s: sending stdin text to C:\Users\Coder\AppData\Roaming\cabal\bin\ghc-mod
-0.234s: C:\Users\Coder\AppData\Roaming\cabal\bin\ghc-mod ended with code 0
-0.232s: C:\Users\Coder\AppData\Roaming\cabal\bin\ghc-mod ended with code 0
-0.007s: ghc-mod said: ghc-mod: <stdin>: hGetLine: invalid argument (invalid byte sequence)
-0.004s: C:\Users\Coder\AppData\Roaming\cabal\bin\ghc-mod ended with code 1
-0.001s: code 1: ghc-mod: <stdin>: hGetLine: invalid argument (invalid byte sequence)

code 1: ghc-mod: <stdin>: hGetLine: invalid argument (invalid byte sequence)

    at module.exports.Util.mkError (file:///C:/Users/Coder/.atom/packages/haskell-ghc-mod/lib/util.coffee:154:15)
    at BufferedProcess.exit (file:///C:/Users/Coder/.atom/packages/haskell-ghc-mod/lib/ghc-mod/ghc-modi-process-real.coffee:117:20)
    at triggerExitCallback (C:\Users\Coder\AppData\Local\atom\app-1.5.0-beta3\resources\app.asar\src\buffered-process.js:213:47)
    at C:\Users\Coder\AppData\Local\atom\app-1.5.0-beta3\resources\app.asar\src\buffered-process.js:227:18
    at Socket.<anonymous> (C:\Users\Coder\AppData\Local\atom\app-1.5.0-beta3\resources\app.asar\src\buffered-process.js:98:18)
    at emitOne (events.js:82:20)
    at Socket.emit (events.js:169:7)
    at Pipe._onclose (net.js:469:12)

Commands

Config

{
  "core": {},
  "haskell-ghc-mod": {
    "debug": true,
    "enableGhcModi": false,
    "ghcModPath": "C:\\Users\\Coder\\AppData\\Roaming\\cabal\\bin\\ghc-mod",
    "ghcModiPath": "C:\\Users\\Coder\\AppData\\Roaming\\cabal\\bin\\ghc-modi",
    "onChangeLint": true
  }
}

Installed Packages

# User
autocomplete-haskell, v0.6.2
haskell-ghc-mod, v1.6.4
ide-haskell, v1.3.9
ide-haskell-cabal, v1.0.0
language-haskell, v1.5.2

# Dev
No dev packages

[lierdakil]

This issue seems to be Windows-specific, since it works fine on POSIX. I think default terminal encoding is to blame. If you have any idea on how to force it with child_process, please share.

[soiamsoNG]

I do some search later, since it just use a getLine function without set a TextEncoding in function getFileSourceFromStdin so it will use system default code page as decoder, but not UTF8 as this package expect.

I think there is three option, but not sure which is better one

1. force ghc-mod interactive mode just can accept UTF8 by hard code in it
2. Add optional args in the interactive mode for codec.
3. Change this package to use code page facility in Windows, I am not sure if node.js can do this. But GHC I am sure it can by mkTextEncoding.

[lierdakil]

Most Haskell sources are UTF-8, and I think GHC complains if it's not, so it pretty much rules out (2) and (3). Which leaves (1). Fixing it in ghc-mod will take a while though, since maintainer doesn't have much time at the moment, so I would opt for a short-term solution until it's fixed in ghc-mod.

[lierdakil]

I could probably run chcp 65001 on ide-haskell start, but I'm not entirely sure it will help. Could you test it out by adding

require('child_process').execSync('chcp 65001')

to your init script?

[soiamsoNG]

@lierdakil could you give a more specify insert point of below code, so i can test

require('child_process').execSync('chcp 65001')

BTW, In the same time i modify getFileSourceFromStdin with utf8, it run well and go ahead to another point and i find you already improve that after ghc-mod 5.5 Set file encoding to utf8 in Lint #724

[lierdakil]

@soiamsoNG, in Atom, open 'Edit' -> 'Open your Init Script'. Paste require('child_process').execSync('chcp 65001'), restart Atom.

[lierdakil]

Oops. Wrong issue.

[soiamsoNG]

after add the code in init got below warning from atom, and no effect

Failed to load C:\Users\Coder.atom\init.coffee EPERM: operation not permitted, write

I try add chcp 65001 in the spawn code in function of runModCmd in this package

process = new BufferedProcess
        command: "chcp 65001&".concat modPath

it can go further but, ghc-mod can read correctly but all it return back is as below �ļ�����Ŀ¼���������﷨����ȷ��

Atom Version: 1.5.0-beta3 System: Unknown Windows Version Thrown From: haskell-ghc-mod package, v1.6.4

Stack Trace

Haskell-ghc-mod: ghc-mod interactive command info failed with error code 1

At caps: {"version":[5,5,0,0],"legacyInteractive":true,"fileMap":true,"rootExec":false,"quoteArgs":true}
URI: C:\Users\Coder\Desktop\test.hs
Args: 
message: �ļ�����Ŀ¼���������﷨����ȷ��

log:
-0.055s: Trying to run ghc-modi in C:\Users\Coder\Desktop
-0.054s: Failed. Falling back to ghc-mod
-0.052s: running C:\Users\Coder\AppData\Roaming\cabal\bin\ghc-mod --map-file,C:\Users\Coder\Desktop\test.hs,info,C:\Users\Coder\Desktop\test.hs, with options.cwd = C:\Users\Coder\Desktop,options.env = [object Object],options.encoding = utf-8
-0.036s: sending stdin text to C:\Users\Coder\AppData\Roaming\cabal\bin\ghc-mod
-0.036s: import System.IO

main :: IO ()
main = do
  let z = "输入法a"
  hSetEncoding stdout utf8
  putStrLn z

-0.036s: info
-0.004s: ghc-mod said: �ļ�����Ŀ¼���������﷨����ȷ��
-0.001s: C:\Users\Coder\AppData\Roaming\cabal\bin\ghc-mod ended with code 1
0s: code 1: �ļ�����Ŀ¼���������﷨����ȷ��

code 1: �ļ�����Ŀ¼���������﷨����ȷ��

    at module.exports.Util.mkError (file:///C:/Users/Coder/.atom/packages/haskell-ghc-mod/lib/util.coffee:154:15)
    at BufferedProcess.exit (file:///C:/Users/Coder/.atom/packages/haskell-ghc-mod/lib/ghc-mod/ghc-modi-process-real.coffee:117:20)
    at triggerExitCallback (C:\Users\Coder\AppData\Local\atom\app-1.5.0-beta3\resources\app.asar\src\buffered-process.js:213:47)
    at ChildProcess.<anonymous> (C:\Users\Coder\AppData\Local\atom\app-1.5.0-beta3\resources\app.asar\src\buffered-process.js:235:18)
    at emitTwo (events.js:87:13)
    at ChildProcess.emit (events.js:172:7)
    at Process.ChildProcess._handle.onexit (internal/child_process.js:200:12)

Commands

     -8:17 core:paste (atom-text-editor.editor.is-focused)
     -7:48.3.0 core:copy (atom-text-editor.editor.is-focused)

Config

{
  "core": {},
  "haskell-ghc-mod": {
    "debug": true,
    "enableGhcModi": false,
    "ghcModPath": "C:\\Users\\Coder\\AppData\\Roaming\\cabal\\bin\\ghc-mod",
    "ghcModiPath": "C:\\Users\\Coder\\AppData\\Roaming\\cabal\\bin\\ghc-modi"
  }
}

Installed Packages

# User
autocomplete-haskell, v0.6.2
haskell-ghc-mod, v1.6.4
hex, v0.6.2
ide-haskell, v1.3.9
ide-haskell-cabal, v1.0.0
language-haskell, v1.5.2

# Dev
No dev packages

[soiamsoNG]

Seems the issue need re-title to "Fail communicate with ghc-mod, when ghc-mod runing under non UTF-8 enviroment" or "Can't Encode output as system wide" and i think the last one is lacking feature in Atom default library. could iconv-lite help?

[lierdakil]

Okay, I created a PR on ghc-mod repo. I think I'll merge it into master provided I didn't break any tests.

[soiamsoNG]

Thanks for your promptly help, I should give you more info since I find another un-forcing encoding Handler in the ghc-mod code. and that will cause following Lint or Check step will get an un-expect encoding temp file(in system wide encoding, when haskell-ghc-mod expect utf-8 temp file), and will display in the Error panel and Lint panel, with encoding error message.

loadMappedFileSource :: IOish m
                     => FilePath -- ^ \'from\', file that will be mapped
                     -> String -- ^ \'src\', source
                     -> GhcModT m ()
loadMappedFileSource from src = do
  tmpdir <- cradleTempDir `fmap` cradle
  to <- liftIO $ do
    (fn, h) <- openTempFile tmpdir (takeFileName from)
    hPutStr h src
    hClose h
    return fn
  loadMappedFile' from to True

[lierdakil]

Oh. Thanks for pointing that out.

[lierdakil]

Should be fixed upstream.