zopencommunity / perlport

Perl programming language
Apache License 2.0
6 stars 3 forks source link

Perl spits out warning about locales #73

Open MikeFultonDev opened 1 year ago

MikeFultonDev commented 1 year ago

The texinfo test suite has a number of failures due to a message about locales. See: https://github.com/ZOSOpenTools/texinfoport/issues/17 for details.

@IgorTodorovskiIBM not sure if this is related to recent changes in Perl or not - perhaps an env var isn't set right now - but this could always have been there (new tests now running).

MikeFultonDev commented 1 year ago

Digging in a little bit, it appears to be because 'ispunct' is giving the wrong answers for the UTF8 locale. Here's a testcase that shows when the test is built in 'ASCII-mode' ispunct gets the wrong answer for the UTF-8 locale and when built in 'EBCDIC-mode' the setlocale to the UTF8 locale fails.

test>clang -fzos-le-char-mode=ascii punctest.c

test>./a.out

C: ispunct('#'):256
UTF-8: ispunct('#'):0

test>clang punctest.c test>./a.out

C: ispunct('#'):16
Unable to set locale to UTF-8

test>cat punctest.c

#include <ctype.h>
#include <stdio.h>
#include <locale.h>

int main() {
  printf("C: ispunct('#'):%d\n", ispunct((unsigned char) ('#')));
  if (!setlocale(LC_ALL, "en_US.UTF-8")) {
    fprintf(stderr, "Unable to set locale to UTF-8\n");
    return 8;
  }
  printf("UTF-8: ispunct('#'):%d\n", ispunct((unsigned char) ('#')));
  return 0;
}
MikeFultonDev commented 1 year ago

Defining your own locale: https://www.ibm.com/docs/en/zos/3.1.0?topic=lu-under-zos-shell and https://www.ibm.com/docs/en/zos/3.1.0?topic=descriptions-localedef-define-locale-environment and https://www.ibm.com/docs/en/zos/3.1.0?topic=lsf-lc-ctype-category

MikeFultonDev commented 1 year ago

Creating a custom locale: https://www.ibm.com/docs/en/zos/3.1.0?topic=ilcs-customizing-locale