search

IBM / ibmi-oss-issues

Important resources for anyone interested in open source on IBM i
Creative Commons Zero v1.0 Universal
13 stars 0 forks source link

Environment variable QIBM_PASE_CCSID will not be considered in bash version 5.1 anymore #14

Closed cobol400 closed 2 months ago

cobol400 commented 2 years ago

After updating bash from version 4.4 to 5.1 the environment variable QIBM_PASE_CCSID will not be considered anymore. We use this variable to ensure that new created files in IFS have CCSID 1252 instead of 1208. We are using IBM i 7.4.

After updating bash to version 5.1 all new created IFS file have CCSID 1208 instead of the value of the environment variable QIBM_PASE_CCSID.

As a workaround we downgraded bash to version 4.4: yum downgrade bash-4.4-6.ppc64

You can test with the following commands: ENDTCPSVR *SSHD ADDENVVAR ENVVAR(PASE_LANG) VALUE(POSIX) LEVEL(*SYS) ADDENVVAR ENVVAR(QIBM_PASE_CCSID) VALUE(1252) LEVEL(*SYS) STRTCPSVR *SSHD

Then connect to IBM i via SSH client (e.g. putty) and be sure that the path /QOpenSys/pkgs/bin is set. touch testfile.txt attr -p testfile.txt ccsid

attr will show CCSID=1252 using bash 4.4 and CCSID=1208 using bash 5.1.

ThePrez commented 2 years ago

1208 is often the preferred and correct behavior. As you've discovered, some changes went into bash 5.1 to fix bugs related to non-UTF8 behavior. The result is that UTF-8 is chosen in more cases, by design. One such case is when the "POSIX" locale is used. You can try setting PASE_LANG to EN_US instead, but I'm not sure that'll work.

What is your use case for needing 1252?

cobol400 commented 2 years ago

The content of our source files have the encoding 1252. When we checkout the sources via git, the files CCSID attribute should also have the CCSID 1252. With file CCSID 1208 and content CCSID 1252 compiling will abort when the source for example includes german umlauts which has different hex values between CCSID 1252 and 1208.

I testet PASE_LANG EN_US but it also sets CCSID 1208.

cobol400 commented 2 years ago

ADDENVVAR ENVVAR(PASE_DEFAULT_UTF8) VALUE(N) LEVEL(*SYS) does also have no impact to the files CCSID.

cobol400 commented 2 years ago

CCSID as an attribute of a file is specific to IBM i. Could you please clarify why fixes related to non-UTF8 behavior lead to disabling the effect of the environment variable QIBM_PASE_CCSID on the CCSID of newly created files? If QIBM_PASE_CCSID interferes with the locale handling in general would it be possible to control the CCSID of newly created files in another way (e.g. another environment variable)?

ThePrez commented 2 years ago

There are a lot of variables at play here. It would be great if git tracked file CCSIDs but that is not possible with how git works.

Whenever a PASE program (or at least, an open source program in PASE) creates a new file, it will be created/tagged with the current PASE CCSID, as you have unfortunately discovered.

We may be able to make changes to help workaround what you are seeing, either through a bash builtin or a change to the bash startup logic. Please stay tuned....

cobol400 commented 1 year ago

Hello, any news about the issue?

cobol400 commented 10 months ago

@ThePrez Do you know when there will be new bash version available which supports QIBM_PASE_CCSID (or something like that) again?

cobol400 commented 7 months ago

I see that there is a new version of bash 5.2. Is there anything new that will fix this issue?

kadler commented 7 months ago

bash 5.2 no longer sets the PASE CCSID automatically, but libc still does. If you are on 7.5, you can apply PTF SI84164 which will prevent libc from changing the PASE CCSID automatically too.