Make Haddock fully UTF-8 compliant and adapt the documentation accordingly

jeltsch commented 4 months ago

Currently, [the subsection on SGML-style character references][character-references] in the documentation still says the following:

Although Haskell source files may contain any character from the Unicode character set, the encoding of these characters as bytes varies between systems, so that only source files restricted to the ASCII character set are portable.

However, it seems that Haddock 2.30.0 mostly supports Haskell code and documentation with non-ASCII characters encoded in UTF-8. I have only spotted the following problems when running cabal haddock under GNU/Linux with the locale set to C:

A non-ASCII character in the synopsis field of the Cabal file causes an error message of the following form:

haddock: internal error: ⟨path⟩/haddock-response⟨number⟩-⟨number⟩.txt: hGetContents: invalid argument (cannot decode byte sequence starting from ⟨number⟩)
A non-ASCII character in the identifier of an undocumented entity causes, during the attempt to output a corresponding warning, an error message of the following form:

haddock: internal error: : commitBuffer: invalid argument (cannot encode character ⟨character⟩)

I think that it would be good to fix these and any other such deficiencies in UTF-8 support and then update the above-mentioned section of the documentation accordingly. Does this sound sensible?

[character-references]: https://haskell-haddock.readthedocs.io/v2.20/markup.html#character-references "Character References"

Kleidukos commented 4 months ago

Hi, thank you for this ticket, but Haddock now lives full-time in the GHC repository! Let's talk about this on https://gitlab.haskell.org/ghc/ghc/ :)

jeltsch commented 4 months ago

The haddock package on Cabal still mentions this here as the bug tracker, and the Haddock documentation even says that Haddock’s code is maintained in a darcs repository (and doesn’t mention where to report bugs).

Kleidukos commented 4 months ago

I am fully aware, thank you very much. This migration has happened after the latest release.

jeltsch commented 4 months ago

Issue filed in the GHC GitLab space: https://gitlab.haskell.org/ghc/ghc/-/issues/24917.

Kleidukos commented 4 months ago

Fantastic.

haskell / haddock

Make Haddock fully UTF-8 compliant and adapt the documentation accordingly #1636