search

Perl / perl5

🐪 The Perl programming language
https://dev.perl.org/perl5/
Other
1.95k stars 554 forks source link

POSIX::readdir is not POSIX compliant #12522

Closed p5pRT closed 11 years ago

p5pRT commented 11 years ago

Migrated from rt.perl.org#115532 (status was 'rejected')

Searchable as RT115532$

p5pRT commented 11 years ago

From perl-diddler@tlinx.org

Created by perl-diddler@tlinx.org

The POSIX​::readdir call isn't POSIX compliant.

It doesn't return the minimal data items required by posix.

The library readir page says​:   Only the fields d_name and d_ino are specified in POSIX.1-2001. The   remaining fields are available on many\, but not all systems.

Neither perl's POSIX nor it's builtin return the inode number\, making it impossible to names to a array of inodes without doing per-file lookups which require opening each inode for the additional information returned (as in a stat call).

Doing that a couple of million times adds a huge amount of per-file latency in looking up inode information -- especially when some file systems allow dumping all inodes in bulk in 1 system call.

Perl Info ``` Flags: category=library severity=critical module=POSIX This perlbug was built using Perl 5.16.1 - Mon Sep 10 13:51:31 UTC 2012 It is being executed now by Perl 5.16.1 - Mon Sep 10 13:45:59 UTC 2012. Site configuration information for perl 5.16.1: Configured by abuild at Mon Sep 10 13:45:59 UTC 2012. Summary of my perl5 (revision 5 version 16 subversion 1) configuration: Platform: osname=linux, osvers=3.4.6-2.10-default, archname=x86_64-linux-thread-multi uname='linux build35 3.4.6-2.10-default #1 smp thu jul 26 09:36:26 utc 2012 (641c197) x86_64 x86_64 x86_64 gnulinux ' config_args='-ds -e -Dprefix=/usr -Dvendorprefix=/usr -Dinstallusrbinperl -Dusethreads -Di_db -Di_dbm -Di_ndbm -Di_gdbm -Dd_dbm_open -Duseshrplib=true -Doptimize=-fmessage-length=0 -O2 -Wall -D_FORTIFY_SOURCE=2 -fstack-protector -funwind-tables -fasynchronous-unwind-tables -g -Wall -pipe -Accflags=-DPERL_USE_SAFE_PUTENV -Dotherlibdirs=/usr/lib/perl5/site_perl' hint=recommended, useposix=true, d_sigaction=define useithreads=define, usemultiplicity=define useperlio=define, d_sfio=undef, uselargefiles=define, usesocks=undef use64bitint=define, use64bitall=define, uselongdouble=undef usemymalloc=n, bincompat5005=undef Compiler: cc='cc', ccflags ='-D_REENTRANT -D_GNU_SOURCE -DPERL_USE_SAFE_PUTENV -fno-strict-aliasing -pipe -fstack-protector -D_LARGEFILE_SOURCE -D_FILE_OFFSET_BITS=64', optimize='-fmessage-length=0 -O2 -Wall -D_FORTIFY_SOURCE=2 -fstack-protector -funwind-tables -fasynchronous-unwind-tables -g -Wall -pipe', cppflags='-D_REENTRANT -D_GNU_SOURCE -DPERL_USE_SAFE_PUTENV -fno-strict-aliasing -pipe -fstack-protector' ccversion='', gccversion='4.7.1 20120723 [gcc-4_7-branch revision 189773]', gccosandvers='' intsize=4, longsize=8, ptrsize=8, doublesize=8, byteorder=12345678 d_longlong=define, longlongsize=8, d_longdbl=define, longdblsize=16 ivtype='long', ivsize=8, nvtype='double', nvsize=8, Off_t='off_t', lseeksize=8 alignbytes=8, prototype=define Linker and Libraries: ld='cc', ldflags =' -L/usr/local/lib64 -fstack-protector' libpth=/lib64 /usr/lib64 /usr/local/lib64 libs=-lm -ldl -lcrypt -lpthread perllibs=-lm -ldl -lcrypt -lpthread libc=/lib64/libc-2.16.so, so=so, useshrplib=true, libperl=libperl.so gnulibc_version='2.16' Dynamic Linking: dlsrc=dl_dlopen.xs, dlext=so, d_dlsymun=undef, ccdlflags='-Wl,-E -Wl,-rpath,/usr/lib/perl5/5.16.1/x86_64-linux-thread-multi/CORE' cccdlflags='-fPIC', lddlflags='-shared -L/usr/local/lib64 -fstack-protector' Locally applied patches: @INC for perl 5.16.1: /home/law/bin/lib /usr/lib/perl5/site_perl/5.16.1/x86_64-linux-thread-multi /usr/lib/perl5/site_perl/5.16.1 /usr/lib/perl5/vendor_perl/5.16.1/x86_64-linux-thread-multi /usr/lib/perl5/vendor_perl/5.16.1 /usr/lib/perl5/5.16.1/x86_64-linux-thread-multi /usr/lib/perl5/5.16.1 /usr/lib/perl5/site_perl/5.16.1/x86_64-linux-thread-multi /usr/lib/perl5/site_perl/5.16.1 /usr/lib/perl5/site_perl /usr/lib/perl5/vendor_perl/5.16.0/x86_64-linux-thread-multi /usr/lib/perl5/vendor_perl/5.16.0 /usr/lib/perl5/vendor_perl /usr/lib/perl5/site_perl . Environment for perl 5.16.1: HOME=/home/law LANG=en_US.UTF-8 LANGUAGE (unset) LC_COLLATE=C LC_CTYPE=en_US.UTF-8 LD_LIBRARY_PATH (unset) LOGDIR (unset) PATH=.:/sbin:/usr/local/sbin:/home/law/bin:/usr/local/bin:/usr/bin:/bin:/usr/bin/X11:/usr/X11R6/bin:/usr/games:/opt/kde3/bin:/usr/lib/mit/bin:/usr/lib/mit/sbin:/usr/lib/qt3/bin:/usr/sbin:/etc/local/func_lib:/home/law/lib:/home/law/bin/lib PERL5OPT=-CSA -I/home/law/bin/lib PERL_BADLANG (unset) SHELL=/bin/bash ```
p5pRT commented 11 years ago

From @Leont

On Wed\, Oct 31\, 2012 at 12​:50 AM\, Linda Walsh \perlbug\-followup@​perl\.org wrote​:

The POSIX​::readdir call isn't POSIX compliant.

It doesn't return the minimal data items required by posix.

The library readir page says​: Only the fields d_name and d_ino are specified in POSIX.1-2001. The remaining fields are available on many\, but not all systems.

Neither perl's POSIX nor it's builtin return the inode number\, making it impossible to names to a array of inodes without doing per-file lookups which require opening each inode for the additional information returned (as in a stat call).

Doing that a couple of million times adds a huge amount of per-file latency in looking up inode information -- especially when some file systems allow dumping all inodes in bulk in 1 system call.

d_ino is part of the XSI extensions\, which means it's not only not strictly required (though that's a bad excuse)\, but also that its standardization probably younger than POSIX.pm (I wish archeology for such things was as easier so I could be sure). It's also not very portable. I would recommend solving this issue on CPAN.

Leon

p5pRT commented 11 years ago

The RT System itself - Status changed from 'new' to 'open'

p5pRT commented 11 years ago

From @jkeenan

On Tue Oct 30 16​:50​:51 2012\, LAWalsh wrote​:

This is a bug report for perl from perl-diddler@​tlinx.org\, generated with the help of perlbug 1.39 running under perl 5.16.1.

----------------------------------------------------------------- [Please describe your issue here]

The POSIX​::readdir call isn't POSIX compliant.

We don't make any claim that the POSIX module is "POSIX compliant."

What 'perldoc POSIX' does say in its DESCRIPTION is​:

"The POSIX module permits you to access all (or nearly all) the standard POSIX 1003.1 identifiers. Many of these identifiers have been given Perl-ish interfaces."

It doesn't return the minimal data items required by posix.

The POSIX readdir function returns a struct\, which\, by definition\, is not a Perl-ish interface.

Neither perl's POSIX nor it's builtin return the inode number\, making it impossible to names to a array of inodes without doing per-file lookups which require opening each inode for the additional information returned (as in a stat call).

Perhaps\, as another poster has suggested\, you should contribute something to CPAN to solve this problem.

As there's no real bug here\, I recommend this ticket be closed.

Thank you very much. Jim Keenan

p5pRT commented 11 years ago

From Mark@Overmeer.net

* James E Keenan via RT (perlbug-followup@​perl.org) [121102 16​:24]​:

Perhaps\, as another poster has suggested\, you should contribute something to CPAN to solve this problem.

POSIX​::1003 welcomes such contributions. --   MarkOv

  Mark Overmeer MSc MARKOV Solutions   Mark@​Overmeer.net solutions@​overmeer.net http​://Mark.Overmeer.net http​://solutions.overmeer.net

p5pRT commented 11 years ago

From @jkeenan

On Fri Nov 02 12​:23​:05 2012\, Mark@​Overmeer.net wrote​:

* James E Keenan via RT (perlbug-followup@​perl.org) [121102 16​:24]​:

Perhaps\, as another poster has suggested\, you should contribute something to CPAN to solve this problem.

POSIX​::1003 welcomes such contributions.

Closing ticket\, as there's a better course of action open on CPAN.

Thank you very much. Jim Keenan

p5pRT commented 11 years ago

@jkeenan - Status changed from 'open' to 'rejected'