Package build error due to unicode

[bos]

(Imported from Trac #626, reported by @basvandijk on 2010-01-26)

The following packages don't build on hackage due to the same error:

• http://hackage.haskell.org/package/regions-0.3
• http://hackage.haskell.org/package/ftdi-0.1

The log contains the following error:

haddock: internal Haddock or GHC error: dist/doc/html/regions/regions.txt: commitAndReleaseBuffer: invalid argument (Invalid or incomplete multibyte or wide character)

I think the .txt file is generated because of the haddock --hoogle flag.

Note that these package make heavy use of Unicode syntax and symbols.

Note that cabal haddock works perferctly on my local system (tested with ghc-6.10.4 + haddock-2.5 + cabal-install-0.6.2 and ghc-6.12.1 + haddock-2.6.0 + cabal-install-0.8.0)

[bos]

(Imported comment by ross on 2010-01-26)

needed to build docs in a UTF8 locale.

[bos]

(Imported comment by @nomeata on 2010-03-18)

I think this is a bug; the behaviour of "Setup haddock" should not depend on the locale of the user; this did bite the Debian package maintainers: http://bugs.debian.org/645573

I guess this can be fixed by a simple call to hSetEncoding.

[bos]

(Imported comment by ross on 2011-10-18)

OK, that would need to be done in the library.

[bos]

(Imported comment by @nomeata on 2011-10-18)

I am currently testing this patch, and will report if it works:

--- ghc-7.0.4.orig/libraries/Cabal/Distribution/Simple/Haddock.hs
+++ ghc-7.0.4/libraries/Cabal/Distribution/Simple/Haddock.hs
@@ -109,7 +109,7 @@ import Data.Maybe    ( fromMaybe, listTo
 import System.FilePath((</>), (<.>), splitFileName, splitExtension,
                        normalise, splitPath, joinPath)
-import System.IO (hClose, hPutStrLn)
+import System.IO (hClose, hPutStrLn, hSetEncoding, utf8)
 import Distribution.Version
 -- Types
@@ -380,6 +380,7 @@ renderArgs verbosity version args k = do
   createDirectoryIfMissingVerbose verbosity True outputDir
   withTempFile outputDir "haddock-prolog.txt" $ \prologFileName h -> do
           do
+             hSetEncoding h utf8
              hPutStrLn h $ fromFlag $ argPrologue args
              hClose h
              let pflag = (:[]).("--prologue="++) $ prologFileName

[bos]

(Imported comment by @nomeata on 2011-10-18)

Hmm, this write the prologue, but then haddock fails with the same error.

http://www.haskell.org/haddock/doc/html/ch03s08.html Says that Unicode characters should be escaped. So I guess this should happen here as well. Or just haddock be made unicode aware...

[nomeata]

JFTR, this bug is still present, and still causes problems.

[gbaz]

relevant: https://github.com/haskell/haddock/issues/573