Serious bug of tell() in perl 5.8

[p5pRT]

Migrated from rt.perl.org#18711 (status was 'resolved')

Searchable as RT18711$

[p5pRT]

From tung@turtle.ee.ncku.edu.tw

Created by tung@turtle.ee.ncku.edu.tw

There is a serious bug in tell() in perl 5.8 on RedHat 8.0.

If a file is opened with '>>'\, the offset returned by tell() is 0 instead of the end offset of the file.

If some data is written after the above open()\, the data will be appended at tail correctly\, but the offset return by tell() is the length of written data instead of the real end offset.

Here is a short example

open(F\, ">>somefile"); # somefile is a file with some text in it

$a=tell(F); # return 0 print "$a\n";

print F "end"; # the str will be appended at tail correctly $a=tell(F); # return 3 print "$a\n";

close(F);

Perl Info

``` Flags: category=core severity=high Site configuration information for perl v5.8.0: Configured by bhcompile at Sun Sep 1 23:55:07 EDT 2002. Summary of my perl5 (revision 5.0 version 8 subversion 0) configuration: Platform: osname=linux, osvers=2.4.18-11smp, archname=i386-linux-thread-multi uname='linux daffy.perf.redhat.com 2.4.18-11smp #1 smp thu aug 15 06:41:59 edt 2002 i686 i686 i386 gnulinux ' config_args='-des -Doptimize=-O2 -march=i386 -mcpu=i686 - Dmyhostname=localhost -Dperladmin=root@localhost -Dcc=gcc -Dcf_by=Red Hat, Inc. -Dinstallprefix=/usr -Dprefix=/usr -Darchname=i386-linux - Dvendorprefix=/usr -Dsiteprefix=/usr -Duseshrplib -Dusethreads -Duseithreads - Duselargefiles -Dd_dosuid -Dd_semctl_semun -Di_db -Ui_ndbm -Di_gdbm - Di_shadow -Di_syslog -Dman3ext=3pm -Duseperlio -Dinstallusrbinperl - Ubincompat5005 -Uversiononly -Dpager=/usr/bin/less -isr' hint=recommended, useposix=true, d_sigaction=define usethreads=define use5005threads=undef useithreads=define usemultiplicity=define useperlio=define d_sfio=undef uselargefiles=define usesocks=undef use64bitint=undef use64bitall=undef uselongdouble=undef usemymalloc=n, bincompat5005=undef Compiler: cc='gcc', ccflags ='-D_REENTRANT -D_GNU_SOURCE -fno-strict-aliasing - D_LARGEFILE_SOURCE -D_FILE_OFFSET_BITS=64 -I/usr/include/gdbm', optimize='-O2 -march=i386 -mcpu=i686', cppflags='-D_REENTRANT -D_GNU_SOURCE -fno-strict-aliasing - I/usr/include/gdbm' ccversion='', gccversion='3.2 20020822 (Red Hat Linux Rawhide 3.2-5)', gccosandvers='' intsize=4, longsize=4, ptrsize=4, doublesize=8, byteorder=1234 d_longlong=define, longlongsize=8, d_longdbl=define, longdblsize=12 ivtype='long', ivsize=4, nvtype='double', nvsize=8, Off_t='off_t', lseeksize=8 alignbytes=4, prototype=define Linker and Libraries: ld='gcc', ldflags =' -L/usr/local/lib' libpth=/usr/local/lib /lib /usr/lib libs=-lnsl -lgdbm -ldb -ldl -lm -lpthread -lc -lcrypt -lutil perllibs=-lnsl -ldl -lm -lpthread -lc -lcrypt -lutil libc=/lib/libc-2.2.92.so, so=so, useshrplib=true, libperl=libperl.so gnulibc_version='2.2.92' Dynamic Linking: dlsrc=dl_dlopen.xs, dlext=so, d_dlsymun=undef, ccdlflags='-rdynamic -Wl,- rpath,/usr/lib/perl5/5.8.0/i386-linux-thread-multi/CORE' cccdlflags='-fpic', lddlflags='-shared -L/usr/local/lib' Locally applied patches: @INC for perl v5.8.0: /usr/lib/perl5/5.8.0/i386-linux-thread-multi /usr/lib/perl5/5.8.0 /usr/lib/perl5/site_perl/5.8.0/i386-linux-thread-multi /usr/lib/perl5/site_perl/5.8.0 /usr/lib/perl5/site_perl /usr/lib/perl5/vendor_perl/5.8.0/i386-linux-thread-multi /usr/lib/perl5/vendor_perl/5.8.0 /usr/lib/perl5/vendor_perl . Environment for perl v5.8.0: HOME=/root LANG=C LANGUAGE (unset) LD_LIBRARY_PATH (unset) LOGDIR (unset) PATH=/usr/local/bin:/bin:/usr/bin:/usr/X11R6/bin:/home/tung/bin PERL_BADLANG (unset) SHELL=/bin/bash ```

[p5pRT]

From @andk

On 27 Nov 2002 10​:17​:59 -0000\, "Chung-Kie Tung" (via RT) \perlbug@​perl\.org said​:

  > # New Ticket Created by "Chung-Kie Tung"   > # Please include the string​: [perl #18711]   > # in the subject line of all future correspondence about this issue.   > # \<URL​: http​://rt.perl.org/rt2/Ticket/Display.html?id=18711 >

  > This is a bug report for perl from tung@​turtle.ee.ncku.edu.tw\,   > generated with the help of perlbug 1.34 running under perl v5.8.0.

  > -----------------------------------------------------------------   > [Please enter your report here]

  > There is a serious bug in tell() in perl 5.8 on RedHat 8.0.

  > If a file is opened with '>>'\,   > the offset returned by tell() is 0 instead of the end offset of the file.

  > If some data is written after the above open()\,   > the data will be appended at tail correctly\,   > but the offset return by tell() is the length of written data   > instead of the real end offset.

  > Here is a short example

  > open(F\, ">>somefile"); # somefile is a file with some text in it

  > $a=tell(F); # return 0   > print "$a\n";

  > print F "end"; # the str will be appended at tail correctly   > $a=tell(F); # return 3   > print "$a\n";

  > close(F);

I cannot reproduce this error. I'm running a RedHat 6.2 system that has all relevant components upgraded. My glibc is still 2.2.5\, not 2.2.92. Maybe it's due to that.

Can anybody else with or without RedHat 8.0 reproduce the bug?

I have tried the script as written above and a modified script that runs an initial

  system("echo foo > somefile");

with hundreds of different perls between 5.6.0 and bleadperl. No unexpected result.

-- andreas

[p5pRT]

From guest@guest.guest.xxxxxxxx

Here is a short example

open(F\, ">>somefile"); # somefile is a file with some text in it

$a=tell(F); # return 0 print "$a\n";

print F "end"; # the str will be appended at tail correctly $a=tell(F); # return 3 print "$a\n";

close(F);

I cannot reproduce this error. I'm running a RedHat 6.2 system that has all relevant components upgraded. My glibc is still 2.2.5\, not 2.2.92. Maybe it's due to that.

Can anybody else with or without RedHat 8.0 reproduce the bug?

I reproduce this error! Windows 2000 Advanced server.

I try test on 3 perl version\, and results differents​:

---

1) perl5 (5.0 patchlevel 5 subversion 03) (from Oracle 8.1.7 distribution)

D​:\temp>W​:\oracle\ora9\Apache\perl\5.00503\bin\MSWin32-x86\perl.exe -w a.pl 0 48

D​:\temp>W​:\oracle\ora9\Apache\perl\5.00503\bin\MSWin32-x86\perl.exe -w a.pl 0 51

---

2) perl5 (revision 5 version 6 subversion 1) ActivePerl Build 633

D​:\temp>c​:\perl.633\bin\perl -w a.pl 51 54

D​:\temp>c​:\perl.633\bin\perl -w a.pl 54 57

---

3) perl5 (revision 5 version 8 subversion 0)

D​:\temp>c​:\perl\bin\perl.exe a.pl 0 3

D​:\temp>c​:\perl\bin\perl.exe a.pl 0 3

---

After last operation ( run c​:\perl\bin\perl.exe a.pl) length ('somefile') == 63 bytes.

[p5pRT]

From tokar@tokar.ru

openwebmail installer openwebmail-tool.pl (http​://turtle.ee.ncku.edu.tw/openwebmail/) print warning​:

WARNING!

The perl on your system has serious bug in routine tell()! While openwebmail can work properly with this bug\, other perl application may not function properly and thus cause data loss.

We suggest that you should patch your perl as soon as possible.

[p5pRT]

nick@ing-simmons.net - Status changed from 'new' to 'open'

[p5pRT]

From nick@ing-simmons.net

I dispute that it is "serious" as usually the whole point of opening for append is to allow multiple processes/threads to update file\, in in such cases relying one snapshot of file size as retuned by tell() is risky.

I note (by experiment) that Linux stdio does not handle read/append files like this​:

  open(FOO\,"+>>somefile");   seek(FOO\,0\,0);   my $foo = \;

man pages don't say it cannot be done - so it should work. (It does work with :unix layer).

If reading does not work\, and you have no control over write position due to implied seek then tell() return value is not important.

However\, because :perlio (which is what shows the bug) is supposed to be an improvement on stdio I have "fixed" the bug. There is still a potential race condition between the tell() and whatever is going to use it\, but value is now correct. There are two possible fixes​:   A. tell() does an lseek(fd\,0\,SEEK_END) and uses that as the   logical position of the write-buffer.   B. tell() does a flush\, and then asks for EOF position.

I have chosen (B) as it is "safer" - i.e. window of race condition is smaller. The implied PertlIO_flush() only happens in append case. While I was there I also removed the need (for :perlio) for a seek() between read/write on a read/write stream.

With the fix code like above which does "+>>somefile" behaves as I would expect.

Fixed by //depot/perlio/perlio.c@​18471

[p5pRT]

nick@ing-simmons.net - Status changed from 'open' to 'resolved'

[p5pRT]

From pjsm@fct.unl.pt

[ni-s - Sun Jan 12 10​:51​:10 2003]​:

open(FOO\,"+>>somefile"); seek(FOO\,0\,0); my $foo = \;

man pages don't say it cannot be done - so it should work. (It does work with :unix layer).

Indeed tell() works on Linux if you _explicitly_ open the file with layer :unix ou :stdio\, like​:

open (FOO\,"+>>​:unix"\,"somefile");

and after the first write operation. Is the correct (or expected) behavior of tell()?