t/io/open.t fails on HPUX with PERLIO=stdio

[demerphq]

Description t/io/open.t fails on HPUX with PERLIO=stdio

You can see this in the smoke reports from @Tux eg: https://tux.nl/perl5/smoke/hpux1131/511/cc/mktest.rpt

The code starting at line 3309 in perlio.c is as follows:

        result = PerlSIO_fclose(stdio);
        /* We treat error from stdio as success if we invalidated
           errno may NOT be expected EBADF
         */
        if (invalidate && result != 0) {
            RESTORE_ERRNO;
            result = 0;
        }

The call to PerlSIO_fclose(stdio) returns a non-zero exit code indicating an error, but it does not set ERRNO. You can force ERRNO to be whatever you like before triggering this code and it does not change. IIRC the "invalidate" variable is false when this code is called on HPUX, although this is from a fallable memory a long time after when i was debugging this.

I looked for errata from HPUX about this, but i couldnt find anything. As far as I can tell the underlying system call here (fclose) is broken on HPUX and does not set ERRNO correctly when it returns an error, and/or returns an error when it shouldnt. I found this extremely difficult to debug.

Steps to Reproduce make test

or

#!./perl

BEGIN {
    chdir 't' if -d 't';
    require './test.pl';
    set_up_inc('../lib');
}

$|  = 1;
use warnings;
use Config;

sub ok_cloexec {
    SKIP: {
        skip "no fcntl", 1 unless $Config{d_fcntl};
        my $fd = fileno($_[0]);
        fresh_perl_is(qq(
            print open(F, "+<&=$fd") ? 1 : 0, "\\n";
        ), "0\n", {}, "not inherited across exec");
    }
}

my $Perl = which_perl();
print "My pid is $$\n";
{
    my $proc = <<EOC;
    $Perl -pe "s/^not //"
EOC
    ok( open(my $f, '|-', $proc),    'open |-' );
    diag "Fileno:", fileno($f);
    #ok_cloexec($f);
    my @rows = <$f>;
    my $test = curr_test;
    print $f "not ok $test - piped in\n";
    next_test;

    $test = curr_test;
    print $f "not ok $test - piped in\n";
    next_test;
    diag "Fileno:", fileno($f);
    $! = 23; # force ERRNO to be 23.
    my $close_ok = close($f);
    my $err = $close_ok ? "" : (0+$!).": $!";
    ok( $close_ok,                      '       close pipe' );
    is( $err, "",                       '       close pipe error');
    sleep 1;
    pass('flushing');
}

Expected behavior test should pass.

Perl configuration

./perl -Ilib -V
Summary of my perl5 (revision 5 version 37 subversion 11) configuration:
  Commit id: 275f4e511457b9578602fc7a19a23cde36fe66bb
  Platform:
    osname=hpux
    osvers=11.31
    archname=IA64.ARCHREV_0-thread-multi-LP64-ld
    uname='hp-ux x2 b.11.31 u ia64 1894272509 unlimited-user license '
    config_args='-Duse64bitall -Dusethreads -Duseithreads -Duselongdouble -Doptimize=-g0 -d -Dusedevel -DDEBUGGING'
    hint=recommended
    useposix=true
    d_sigaction=define
    useithreads=define
    usemultiplicity=define
    use64bitint=define
    use64bitall=define
    uselongdouble=define
    usemymalloc=n
    default_inc_excludes_dot=define
  Compiler:
    cc='cc'
    ccflags =' -D_POSIX_C_SOURCE=199506L -D_REENTRANT -Ae -Wp,-H150000 -D_HPUX_SOURCE -Wl,+vnocompatwarnings +DD64 -DDEBUGGING -I/usr/local/include -D_LARGEFILE_SOURCE -D_FILE_OFFSET_BITS=64 '
    optimize='-g0'
    cppflags='-Aa -D__STDC_EXT__ -D_HPUX_SOURCE -D_POSIX_C_SOURCE=199506L -D_REENTRANT -Ae -Wp,-H150000 -D_HPUX_SOURCE -Wl,+vnocompatwarnings +DD64 -DDEBUGGING -I/usr/local/include'
    ccversion='B3910B A.06.28.02'
    gccversion=''
    gccosandvers=''
    intsize=4
    longsize=8
    ptrsize=8
    doublesize=8
    byteorder=87654321
    doublekind=4
    d_longlong=define
    longlongsize=8
    d_longdbl=define
    longdblsize=16
    longdblkind=2
    ivtype='long'
    ivsize=8
    nvtype='long double'
    nvsize=16
    Off_t='off_t'
    lseeksize=8
    alignbytes=16
    prototype=define
  Linker and Libraries:
    ld='/usr/bin/ld'
    ldflags =' +DD64 -L/usr/local/lib -L/usr/lib/hpux64'
    libpth=/usr/local/lib /usr/lib/hpux64 /lib /usr/lib /usr/ccs/lib
    libs=-lcl -lpthread -ldl -lm -lsec -lc
    perllibs=-lcl -lpthread -ldl -lm -lsec -lc
    libc=/usr/lib/hpux64/libc.so
    so=so
    useshrplib=false
    libperl=libperl.a
    gnulibc_version=''
  Dynamic Linking:
    dlsrc=dl_hpux.xs
    dlext=so
    d_dlsymun=undef
    ccdlflags='-Wl,-E -Wl,-B,deferred '
    cccdlflags='+Z'
    lddlflags='-b +vnocompatwarnings -L/usr/local/lib -L/usr/lib/hpux64'

Characteristics of this binary (from libperl): 
  Compile-time options:
    DEBUGGING
    HAS_LONG_DOUBLE
    HAS_STRTOLD
    HAS_TIMES
    MULTIPLICITY
    PERLIO_LAYERS
    PERL_COPY_ON_WRITE
    PERL_DONT_CREATE_GVSV
    PERL_HASH_FUNC_SIPHASH13
    PERL_HASH_USE_SBOX32
    PERL_MALLOC_WRAP
    PERL_OP_PARENT
    PERL_PRESERVE_IVUV
    PERL_TRACK_MEMPOOL
    PERL_USE_DEVEL
    PERL_USE_SAFE_PUTENV
    USE_64_BIT_ALL
    USE_64_BIT_INT
    USE_ITHREADS
    USE_LARGE_FILES
    USE_LOCALE
    USE_LOCALE_COLLATE
    USE_LOCALE_CTYPE
    USE_LOCALE_NUMERIC
    USE_LOCALE_TIME
    USE_LONG_DOUBLE
    USE_PERLIO
    USE_PERL_ATOF
    USE_REENTRANT_API
  Built under hpux
  Compiled at Mar 31 2023 20:11:41
  @INC:
    lib
    /opt/perl5/lib/site_perl/5.37.11/IA64.ARCHREV_0-thread-multi-LP64-ld
    /opt/perl5/lib/site_perl/5.37.11
    /opt/perl5/lib/5.37.11/IA64.ARCHREV_0-thread-multi-LP64-ld
    /opt/perl5/lib/5.37.11

[Tux]

stdio smoking has been disabled on this HP-UX box. All was green after that. Can this issue be closed now?

I /think/ a note in README.hpux would be appropriate. @demerphq ?

[demerphq]

@tux that makes sense to me. Normal builds work, not sure we should care too much about stdio builds given there aren't many people targeting this platform.

[Tux]

AIX suffers the same failure pattern, but I do not know if it is the same cause.

[demerphq]

AIX suffers the same failure pattern, but I do not know if it is the same cause.

That is a bit surprising. Do they share C libraries or compilers or something? If not that suggests there is something beyond a bug in the standard libraries at fault here. My analysis was that there is a bug in fclose(), but it seems unlikely that such a bug would affect two different platforms. We should try to dig deeper into this I think.

[Tux]

Automated smoke report for branch blead 5.38.0 patch 815251bd22ddf4bf221549c96736004652928824 v5.38.0-RC2-2-g815251b
i3: PPC_POWER5 (ppc/1 cpu)
    on        AIX - 5.3.0.0/TL12-05
    using     xlc -q32 version 12.1.0.12
    smoketime 1 day 34 minutes (average 3 hours 4 minutes)

Summary: FAIL(F)

O = OK  F = Failure(s), extended report at the bottom
X = Failure(s) under TEST but not under harness
? = still running or test results not (yet) available
Build failures during:       - = unknown or N/A
c = Configure, m = make, M = make (after miniperl), t = make test-prep

v5.38.0-RC2-2-g815251b  Configuration (common) none
----------- ---------------------------------------------------------
F O O F O O 
F O O F O O -Duse64bitall
F O O F O O -Duseithreads
F O O F O O -Duseithreads -Duse64bitall
| | | | | +- LC_ALL = EN_US.UTF-8 -DDEBUGGING
| | | | +--- PERLIO = perlio -DDEBUGGING
| | | +----- PERLIO = stdio  -DDEBUGGING
| | +------- LC_ALL = EN_US.UTF-8
| +--------- PERLIO = perlio
+----------- PERLIO = stdio 

Locally applied patches:
    RC2
    uncommitted-changes
    SMOKE815251bd22ddf4bf221549c96736004652928824

Testsuite was run only with 'harness'

Failures: (common-args) none
[stdio] 
[stdio] -DDEBUGGING
[stdio] -Duse64bitall
[stdio] -DDEBUGGING -Duse64bitall
[stdio] -Duseithreads
[stdio] -DDEBUGGING -Duseithreads
[stdio] -Duseithreads -Duse64bitall
[stdio] -DDEBUGGING -Duseithreads -Duse64bitall
../t/io/open.t..............................................FAILED
    44, 92

[Leont]

I'm strongly suspecting this is another case of fallout of 80c1f1e45e8ef8c27d170fae7ade41971fe20218, just like #21187