GCC-11 cut test FAIL

[frikiluser]

Hi,

After building toybox using gcc-11.2 on a Debian Sid environment I've found that cut test doesn't PASS.

The output is:

PASS: cut -c japan.txt
FAIL: cut -C test1.txt
echo -ne '' | cut -C -1 "$FILES/utf8/test1.txt"
--- expected    2021-10-27 20:24:55.607278439 +0000
+++ actual      2021-10-27 20:24:55.615278472 +0000
@@ -1 +1 @@
-l̗
+l
make[1]: *** [Makefile:78: tests] Error 1

An hexdump of the diff may help:

2d 6c cc b4 cc 97 cc 9e  cc a0 0a 2b 6c 0a        |-l.........+l.|

Followed build steps:

CFLAGS="-g -O2 -ffile-prefix-map=/build/toybox-0.8.5=. -fstack-protector-strong -Wformat -Werror=format-security -Wdate-time -D_FORTIFY_SOURCE=2 -Wl,-z,relro -Wl,-z,now" make defconfig
make -j12
make tests

Please, let me know if you can't reproduce the issue. I can reproduce the problem in a clean and reproducible environment.

Best regards,

[landley]

The new glibc version is failing to identify combining characters. Elliott and I were just talking about that on the list a few days ago, bionic can't do it when statically linked and it looks like glibc's had a regression? (Does it need a locale set now, or something?)

[enh-google]

This certainly looks similar to the macOS failure when you're in the C locale. I thought we already hard-coded a utf8 locale in this test for that reason though?

On Wed, Oct 27, 2021, 22:55 Rob Landley @.***> wrote:

The new glibc version is failing to identify combining characters. Elliott and I were just talking about that on the list a few days ago, bionic can't do it when statically linked and it looks like glibc's had a regression? (Does it need a locale set now, or something?)

— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub https://github.com/landley/toybox/issues/300#issuecomment-953525123, or unsubscribe https://github.com/notifications/unsubscribe-auth/AMVLEWHIPKULYKG3L7SXN53UJDQWFANCNFSM5G3IRSHA . Triage notifications on the go with GitHub Mobile for iOS https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675 or Android https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub.

[landley]

What does it do if they haven't got a utf8 locale installed?

[enh-google]

No idea, I'm a default install kind of guy 😃 but, yeah, that sounds plausible. Ask the submitter for an strace?

On Wed, Oct 27, 2021, 23:05 Rob Landley @.***> wrote:

What does it do if they haven't got a utf8 locale installed?

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/landley/toybox/issues/300#issuecomment-953529585, or unsubscribe https://github.com/notifications/unsubscribe-auth/AMVLEWCQDJKP7OIKLEVQS3DUJDR4JANCNFSM5G3IRSHA . Triage notifications on the go with GitHub Mobile for iOS https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675 or Android https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub.

[landley]

I'm trying to figure out how we'd add a test guard for this in the test suite. "locale -a" is a glibc command, I wouldn't expect musl or bionic to have it. Can't really build a test C program in the test suite. (It should be able to run on an existing toybox binary in a target system, without a compiler.) Hmmm...

[enh-google]

needs utf8 locale for utf8 tests to pass.

[landley]

I may still need to track this issue: devuan caligula has gcc 8.3 with glibc 2.28 so my TEST_HOST isn't currently hitting this, but may when I upgrade. If this is another permanent gnu regression ala "static builds used to be able to getpwnam() but lost that ability in a version 'upgrade'" or a distro regression ala "all red hat systems could build a statically linked hello world until they moved libc.a into a separate package that isn't installed by default", then I need to at least be able to probe for the breakage and disable the test on systems where they did this.

[frikiluser]

FTR: 0.8.8 works in my building environment (Debian Sid)

Thanks!