Fix printing non-(ASCII/UTF-8) strings

[ghost]

As reported in FreeBSD's PR241491, ps(1) using libxo does not display the date in uk_UA.KOI8-U locale when invoked as ps -o lstart and prints empty lines instead. To fix it we need to assume that %s is passed in current locale encoding.

• use XF_ENC_LOCALE for 's' format character
• pass "" to setlocale() so that if the consumer actually called setlocale() itself we don't overwrite already set locale; current order of environment variables differs from at least FreeBSD's setlocale() where the following order is used: LCALL LC* LANG

[philshafer]

On Oct 25, 2019, at 6:06 PM, Yuri Pankov notifications@github.com wrote:

To fix it we need to assume that %s is passed in current locale encoding.

The “%s” format does not mean “current locale”. If ps wants that, it needs to use “%hs”. “%s” means UTF-8:

https://libxo.readthedocs.io/en/latest/field-formatting.html#utf-8-and-locale-strings https://libxo.readthedocs.io/en/latest/field-formatting.html#utf-8-and-locale-strings

Re: the admittedly strange arg to setlocale(), my testing found that “” didn’t pick up the environment settings, so I coded it explicitly. If the order doesn’t follow the FreeBSD specs, I can repair it. Is this order documented somewhere in the FreeBSD docs?

Thanks, Phil

[ghost]

On Oct 25, 2019, at 6:06 PM, Yuri Pankov @.***> wrote: To fix it we need to assume that %s is passed in current locale encoding. The “%s” format does not mean “current locale”. If ps wants that, it needs to use “%hs”. “%s” means UTF-8: https://libxo.readthedocs.io/en/latest/field-formatting.html#utf-8-and-locale-strings https://libxo.readthedocs.io/en/latest/field-formatting.html#utf-8-and-locale-strings

Got it, and sorry, that the first time I looked at libxo specific issue. I'll close this one and will fix the ps instead, thanks!

Re: the admittedly strange arg to setlocale(), my testing found that “” didn’t pick up the environment settings, so I coded it explicitly. If the order doesn’t follow the FreeBSD specs, I can repair it. Is this order documented somewhere in the FreeBSD docs?

Not that I know of, the code for reference: https://svnweb.freebsd.org/base/head/lib/libc/locale/setlocale.c?revision=326025&view=markup#l285.

And if you could provide a bit more info on "" not working (possibly as PR), that would be helpful!

[philshafer]

On Oct 26, 2019, at 11:30 AM, Phil Shafer phil@juise.org wrote:

Re: the admittedly strange arg to setlocale(), my testing found that “” didn’t pick up the environment settings, so I coded it explicitly. If the order doesn’t follow the FreeBSD specs, I can repair it. Is this order documented somewhere in the FreeBSD docs?

Yuri, Please review the attached patch, which mungs the order into the FreeBSD one and avoids it completely for FreeBSD.

Thanks, Phil

————————

+++ b/libxo/libxo.c @@ -624,13 +624,18 @@ xo_init_handle (xo_handle_t xop) if (!xo_locale_inited) { xo_locale_inited = 1; / Only do this once */

• const char cp = getenv("LC_CTYPE"); +#ifdef FreeBSD / Who does The Right Thing */
• const char cp = ""; +#else / FreeBSD */
• const char *cp = getenv("LC_ALL"); if (cp == NULL)
• cp = getenv("LANG");
• cp = getenv("LC_CTYPE"); if (cp == NULL)
• cp = getenv("LC_ALL");
• cp = getenv("LANG"); if (cp == NULL) cp = "C"; / Default for C programs / +#endif / FreeBSD /

• (void) setlocale(LC_CTYPE, cp); }