search

SWI-Prolog / issues

Dummy repository for issue tracking
7 stars 3 forks source link

Encoding always set to text regardless of the LANG environment variable #90

Open xh4 opened 5 years ago

xh4 commented 5 years ago

SWI-Prolog version 8.0.2 on Windows 10

On Windows, the encoding of prolog always set to text regardless of the LANG environment variable.

?- getenv('LANG', E).
E = 'en_US.UTF-8'.
?- current_prolog_flag(encoding, V).
V = text.
JanWielemaker commented 5 years ago

The encoding flag initialization is derived from LD->encoding which is initialized at is at https://github.com/SWI-Prolog/swipl-devel/blob/14668201036cb8d6fd9d5095bb58d6be6e8b6993/src/os/pl-ctype.c#L886

Here we see it only used LC_CTYPE locale. This is a bit shaky. On the one hand we do not rely on the environment variables directly, but let setlocale() figure it out depending on the platform. On the other hand setlocale() returns according to the docs an opaque string. On most (Unix) systems this is simply the final locale name.

I do not know much about Microsoft locale handling. Here is a link that doesn't sound promising.

Note that utf8 merely replaces the rather indirect route through the C library for I/O conversion with its own dedicated UTF-8 routines. If the system does UTF-8 as specified by the locale this should still work as text simply state leave it to the C library.

The docs about LANG are wrong though and I do not really know how to make Windows UTF-8 friendly. If you know, share it and I'll try to make SWI-Prolog cooperate.

For Prolog itself you can set encoding in your swipl.ini or application.