threads->tid incorrect in DESTROY

[p5pRT]

Migrated from rt.perl.org#10046 (status was 'resolved')

Searchable as RT10046$

[p5pRT]

From @lizmat

Created by @lizmat

It seems that threads->tid is reported incorrectly inside a DESTROY. The tid reported is the tid in which the object was originally created.

This basically forces the object to keep a flag itself whether any shared parts of the object are already destroyed or not. And may have implications for modules such as DBI being used in an environment where there are threads active.

Set the severity to "medium"\, but this may have bigger implications which might warrant an increase in severity. ;-(

== tidwrong ===================================================== use threads;

my $async = async { my $foo = Foo->new; my $thread = threads->new( sub {print "in thread ".threads->tid."\n"} ); print "waiting for thread to finish\n"; $thread->join; print "thread now finished\n"; };

$async->join;

package Foo; sub new { bless {}\,shift }; sub DESTROY { print "DESTROYing in thread ".threads->tid."\n"}

  $ perl -w tidwrong   waiting for thread to finish   in thread 2   DESTROYing in thread 1   thread now finished   DESTROYing in thread 1

Perl Info

``` Flags: category=core severity=medium Site configuration information for perl v5.8.0: Configured by liz at Wed Jul 10 18:32:18 CEST 2002. Summary of my perl5 (revision 5.0 version 8 subversion 0 patch 17462) configuration: Platform: osname=linux, osvers=2.4.18, archname=i686-linux-thread-multi uname='linux echt.dijkmat.nl 2.4.18 #8 smp mon mar 25 22:28:36 cet 2002 i686 unknown ' config_args='-de -Dusethreads' hint=recommended, useposix=true, d_sigaction=define usethreads=define use5005threads=undef useithreads=define usemultiplicity=define useperlio=define d_sfio=undef uselargefiles=define usesocks=undef use64bitint=undef use64bitall=undef uselongdouble=undef usemymalloc=n, bincompat5005=undef Compiler: cc='cc', ccflags ='-D_REENTRANT -D_GNU_SOURCE -fno-strict-aliasing -I/usr/local/include -D_LARGEFILE_SOURCE -D_FILE_OFFSET_BITS=64 -I/usr/include/gdbm', optimize='-O2', cppflags='-D_REENTRANT -D_GNU_SOURCE -fno-strict-aliasing -I/usr/local/include -I/usr/include/gdbm' ccversion='', gccversion='2.96 20000731 (Red Hat Linux 7.1 2.96-98)', gccosandvers='' intsize=4, longsize=4, ptrsize=4, doublesize=8, byteorder=1234 d_longlong=define, longlongsize=8, d_longdbl=define, longdblsize=12 ivtype='long', ivsize=4, nvtype='double', nvsize=8, Off_t='off_t', lseeksize=8 alignbytes=4, prototype=define Linker and Libraries: ld='cc', ldflags =' -L/usr/local/lib' libpth=/usr/local/lib /lib /usr/lib libs=-lnsl -lgdbm -ldb -ldl -lm -lpthread -lc -lcrypt -lutil perllibs=-lnsl -ldl -lm -lpthread -lc -lcrypt -lutil libc=/lib/libc-2.2.4.so, so=so, useshrplib=false, libperl=libperl.a gnulibc_version='2.2.4' Dynamic Linking: dlsrc=dl_dlopen.xs, dlext=so, d_dlsymun=undef, ccdlflags='-rdynamic' cccdlflags='-fpic', lddlflags='-shared -L/usr/local/lib' Locally applied patches: DEVEL17462 @INC for perl v5.8.0: /usr/local/lib/perl5/5.8.0/i686-linux-thread-multi /usr/local/lib/perl5/5.8.0 /usr/local/lib/perl5/site_perl/5.8.0/i686-linux-thread-multi /usr/local/lib/perl5/site_perl/5.8.0 /usr/local/lib/perl5/site_perl/5.7.3 /usr/local/lib/perl5/site_perl . Environment for perl v5.8.0: HOME=/home/liz LANG=en_US LANGUAGE (unset) LD_LIBRARY_PATH (unset) LOGDIR (unset) PATH=/usr/local/bin:/bin:/usr/bin:/usr/X11R6/bin:/home/liz/bin PERL_BADLANG (unset) SHELL=/bin/bash ```

[p5pRT]

From [Unknown Contact. See original ticket]

Hah\, this sounds interesting\,

I think you will find it reports the tid of the thread that does the join :-)\, I don't know offhand how to fix it.

Arthur

On fredag\, juli 12\, 2002\, at 05​:31 \, Jarkko Hietaniemi wrote​:

*sigh* Why do people have to do this\, find more and more bugs just before a release...?

----- Forwarded message from Elizabeth Mattijsen \liz@​dijkmat\.nl -----

Subject​: [ID 20020712.002] threads->tid incorrect in DESTROY From​: Elizabeth Mattijsen \liz@​dijkmat\.nl Date​: Fri\, 12 Jul 2002 17​:14​:27 +0200 Message-Id​: \4\.2\.0\.58\.20020712171338\.02556380@​mickey\.dijkmat\.nl To​: perl5-porters@​perl.org

This is a bug report for perl from liz@​dijkmat.nl\, generated with the help of perlbug 1.34 running under perl v5.8.0.

----------------------------------------------------------------- [Please enter your report here]

It seems that threads->tid is reported incorrectly inside a DESTROY. The tid reported is the tid in which the object was originally created.

This basically forces the object to keep a flag itself whether any shared parts of the object are already destroyed or not. And may have implications for modules such as DBI being used in an environment where there are threads active.

Set the severity to "medium"\, but this may have bigger implications which might warrant an increase in severity. ;-(

== tidwrong ===================================================== use threads;

my $async = async { my $foo = Foo->new; my $thread = threads->new( sub {print "in thread ".threads->tid."\n"} ); print "waiting for thread to finish\n"; $thread->join; print "thread now finished\n"; };

$async->join;

package Foo; sub new { bless {}\,shift }; sub DESTROY { print "DESTROYing in thread ".threads->tid."\n"}

$ perl -w tidwrong waiting for thread to finish in thread 2 DESTROYing in thread 1 thread now finished DESTROYing in thread 1

[Please do not change anything below this line] ----------------------------------------------------------------- --- Flags​: category=core severity=medium --- Site configuration information for perl v5.8.0​:

Configured by liz at Wed Jul 10 18​:32​:18 CEST 2002.

Summary of my perl5 (revision 5.0 version 8 subversion 0 patch 17462) configuration​: Platform​: osname=linux\, osvers=2.4.18\, archname=i686-linux-thread-multi uname='linux echt.dijkmat.nl 2.4.18 #8 smp mon mar 25 22​:28​:36 cet 2002 i686 unknown ' config_args='-de -Dusethreads' hint=recommended\, useposix=true\, d_sigaction=define usethreads=define use5005threads=undef useithreads=define usemultiplicity=define useperlio=define d_sfio=undef uselargefiles=define usesocks=undef use64bitint=undef use64bitall=undef uselongdouble=undef usemymalloc=n\, bincompat5005=undef Compiler​: cc='cc'\, ccflags ='-D_REENTRANT -D_GNU_SOURCE -fno-strict-aliasing -I/usr/local/include -D_LARGEFILE_SOURCE -D_FILE_OFFSET_BITS=64 -I/usr/include/gdbm'\, optimize='-O2'\, cppflags='-D_REENTRANT -D_GNU_SOURCE -fno-strict-aliasing -I/usr/local/include -I/usr/include/gdbm' ccversion=''\, gccversion='2.96 20000731 (Red Hat Linux 7.1 2.96-98)'\, gccosandvers='' intsize=4\, longsize=4\, ptrsize=4\, doublesize=8\, byteorder=1234 d_longlong=define\, longlongsize=8\, d_longdbl=define\, longdblsize=12 ivtype='long'\, ivsize=4\, nvtype='double'\, nvsize=8\, Off_t='off_t'\, lseeksize=8 alignbytes=4\, prototype=define Linker and Libraries​: ld='cc'\, ldflags =' -L/usr/local/lib' libpth=/usr/local/lib /lib /usr/lib libs=-lnsl -lgdbm -ldb -ldl -lm -lpthread -lc -lcrypt -lutil perllibs=-lnsl -ldl -lm -lpthread -lc -lcrypt -lutil libc=/lib/libc-2.2.4.so\, so=so\, useshrplib=false\, libperl=libperl.a gnulibc_version='2.2.4' Dynamic Linking​: dlsrc=dl_dlopen.xs\, dlext=so\, d_dlsymun=undef\, ccdlflags='-rdynamic' cccdlflags='-fpic'\, lddlflags='-shared -L/usr/local/lib'

Locally applied patches​: DEVEL17462

--- @​INC for perl v5.8.0​: /usr/local/lib/perl5/5.8.0/i686-linux-thread-multi /usr/local/lib/perl5/5.8.0 /usr/local/lib/perl5/site_perl/5.8.0/i686-linux-thread-multi /usr/local/lib/perl5/site_perl/5.8.0 /usr/local/lib/perl5/site_perl/5.7.3 /usr/local/lib/perl5/site_perl .

--- Environment for perl v5.8.0​: HOME=/home/liz LANG=en_US LANGUAGE (unset) LD_LIBRARY_PATH (unset) LOGDIR (unset) PATH=/usr/local/bin​:/bin​:/usr/bin​:/usr/X11R6/bin​:/home/liz/bin PERL_BADLANG (unset) SHELL=/bin/bash

----- End forwarded message -----

-- $jhi++; # http​://www.iki.fi/jhi/ # There is this special biologist word we use for 'stable'. # It is 'dead'. -- Jack Cohen

[p5pRT]

From @lizmat

At 05​:14 PM 7/12/02 +0200\, you wrote​:

This is a bug report for perl from liz@​dijkmat.nl\, generated with the help of perlbug 1.34 running under perl v5.8.0.

----------------------------------------------------------------- [Please enter your report here]

It seems that threads->tid is reported incorrectly inside a DESTROY. The tid reported is the tid in which the object was originally created.

The same problem seems to exist with the CLONE method.

Liz

[p5pRT]

From @lizmat

At 04​:38 PM 7/21/02 +0200\, Elizabeth Mattijsen wrote​:

At 05​:14 PM 7/12/02 +0200\, you wrote​:

It seems that threads->tid is reported incorrectly inside a DESTROY. The tid reported is the tid in which the object was originally created. The same problem seems to exist with the CLONE method.

With my first attempts at XS\, I found out that the problem is actually even deeper. The entire context is still in the originating thread\, judging from the value of the C getpid() function.

perlmod.pod states​:

"CLONE will be executed once for every package that has it defined (or inherits it). It will be called in the context of the new thread\, so all modifications are made in the new area."

This may be true at some Perl level\, it definitely does not seem to be true at the C level (I have no other explanation for getpid() showing the pid of the originating thread's process under Linux).

Liz

[p5pRT]

From arthur@contiller.se

On måndag\, september 2\, 2002\, at 10​:07 \, Elizabeth Mattijsen wrote​:

With my first attempts at XS\, I found out that the problem is actually even deeper. The entire context is still in the originating thread\, judging from the value of the C getpid() function.

Uhm\, please do not confuse the perl context with the underlying thread context\, in theory we are free to migrate perl threads between underlying OS threads (mod_perl does that)

perlmod.pod states​:

"CLONE will be executed once for every package that has it defined (or inherits it). It will be called in the context of the new thread\, so all modifications are made in the new area."

This may be true at some Perl level\, it definitely does not seem to be true at the C level (I have no other explanation for getpid() showing the pid of the originating thread's process under Linux).

The cloning is done in a new context but in the old os thread.

Arthur

[p5pRT]

From @lizmat

At 11​:53 AM 9/2/02 +0200\, Arthur Bergman wrote​:

This may be true at some Perl level\, it definitely does not seem to be true at the C level (I have no other explanation for getpid() showing the pid of the originating thread's process under Linux). The cloning is done in a new context but in the old os thread.

I assume that's the same for a CLONE {} that I write in XS?

Liz

[p5pRT]

From arthur@contiller.se

On måndag\, september 2\, 2002\, at 12​:04 \, Elizabeth Mattijsen wrote​:

I assume that's the same for a CLONE {} that I write in XS?

Indeed.

Arthur

[p5pRT]

From @lizmat

At 01​:12 PM 9/2/02 +0200\, Arthur Bergman wrote​:

I assume that's the same for a CLONE {} that I write in XS? Indeed.

Would it be possible to get at the new thread id and the getpid() of the new thread in XS from within a CLONE subroutine?

Liz

[p5pRT]

From goldbb2@earthlink.net

Elizabeth Mattijsen wrote​:

At 01​:12 PM 9/2/02 +0200\, Arthur Bergman wrote​:

I assume that's the same for a CLONE {} that I write in XS? Indeed.

Would it be possible to get at the new thread id and the getpid() of the new thread in XS from within a CLONE subroutine?

Within the CLONE method\, the "current thread id" should *be* the "new thread id". Regardless of whether or not doing it in XS.

And the new OS-thread may not yet have been created.

-- tr/`4/ /d\, print "@​{[map --$| ? ucfirst lc : lc\, split]}\,\n" for pack 'u'\, pack 'H*'\, 'ab5cf4021bafd28972030972b00a218eb9720000';

[p5pRT]

From arthur@contiller.se

On måndag\, september 2\, 2002\, at 11​:55 \, Elizabeth Mattijsen wrote​:

At 01​:12 PM 9/2/02 +0200\, Arthur Bergman wrote​:

I assume that's the same for a CLONE {} that I write in XS? Indeed.

Would it be possible to get at the new thread id and the getpid() of the new thread in XS from within a CLONE subroutine?

Not the getpid since the process thread is not created yet.

Arthur

[p5pRT]

From nick.ing-simmons@elixent.com

Elizabeth Mattijsen \liz@​dijkmat\.nl writes​:

At 01​:12 PM 9/2/02 +0200\, Arthur Bergman wrote​:

I assume that's the same for a CLONE {} that I write in XS? Indeed.

Would it be possible to get at the new thread id and the getpid() of the new thread in XS from within a CLONE subroutine?

Not at present. The new thread has not been created at point CLONE is called. We call CLONE to create data area for new thread. As soon a new thread is created it starts running and accessing data so it needs to have somewhere to play.

I can think of two alternatives​: A. Suspend the original thread (to keep it out of to-be-cloned data)   and do CLONE in chiled thread\, then awaken parent thread.   (A bit like doing a vfork rather than fork I guess.)

B. Invent CLONED (say) which is run in child thread before calling   the sub. Maybe we could generalize this to have   PRECLONE { } and POSTCLONE { } lists akin   BEGIN {} CHECK {} INIT {} END {}

Liz -- Nick Ing-Simmons http​://www.ni-s.u-net.com/

[p5pRT]

From @lizmat

At 06​:25 PM 9/2/02 -0400\, Benjamin Goldberg wrote​:

Would it be possible to get at the new thread id and the getpid() of the new thread in XS from within a CLONE subroutine? Within the CLONE method\, the "current thread id" should *be* the "new thread id". Regardless of whether or not doing it in XS.

Well\, I agree. But that is _not_ the case right now.

And the new OS-thread may not yet have been created.

Well\, after some more thinking\, I created a work around that seems to work. By hijacking threads​::new\, you can run a subroutine inside the thread before the actual thread subroutine is started.

Why wasn't CLONE implemented that way?

Liz

[p5pRT]

From @rgarcia

Elizabeth Mattijsen \liz@​dijkmat\.nl wrote​:

Well\, after some more thinking\, I created a work around that seems to work. By hijacking threads​::new\, you can run a subroutine inside the thread before the actual thread subroutine is started.

Do you mean that you subclass threads ? Is it possible to subclass threads ? (BTW thread objects are done this way in Java.)

[p5pRT]

From @lizmat

At 01​:24 PM 9/3/02 +0200\, Rafael Garcia-Suarez wrote​:

Well\, after some more thinking\, I created a work around that seems to work. By hijacking threads​::new\, you can run a subroutine inside the thread before the actual thread subroutine is started. Do you mean that you subclass threads ? Is it possible to subclass threads ? (BTW thread objects are done this way in Java.)

Well\, you _could_ subclass threads.pm\, why not?

But what I was thinking\, was more along the lines of​:

  use threads;   my $new = \&threads​::new;   *threads​::new = sub {   my $class = shift;   my $sub = shift;   * do whatever I want to do inside the thread here   goto &$sub;   };

With this trick I can​:

  - reliably get signals to automatically register in a thread   for Thread​::Status\, which will allow you to obtain a live   report of where each thread is running in the code by just   specifying -MThread​::Status in the command line.

  - allow me to return from anywhere in a thread wthout getting   a message to STDERR (anyone objects to stealing exit() for   this?)\, which I'll probably call Thread​::Exit.

  - lot's of other nice stuff involving events.

Liz

[p5pRT]

From @tux

On Wed 04 Sep 2002 01​:32\, Elizabeth Mattijsen \liz@​dijkmat\.nl wrote​:

At 01​:24 PM 9/3/02 +0200\, Rafael Garcia-Suarez wrote​:

Well\, after some more thinking\, I created a work around that seems to work. By hijacking threads​::new\, you can run a subroutine inside the thread before the actual thread subroutine is started. Do you mean that you subclass threads ? Is it possible to subclass threads ? (BTW thread objects are done this way in Java.)

Well\, you _could_ subclass threads.pm\, why not?

But what I was thinking\, was more along the lines of​:

use threads; my $new = \&threads​::new; *threads​::new = sub { my $class = shift; my $sub = shift; * do whatever I want to do inside the thread here goto &$sub;

And where do you use $new now?

};

With this trick I can​:

- reliably get signals to automatically register in a thread for Thread​::Status\, which will allow you to obtain a live report of where each thread is running in the code by just specifying -MThread​::Status in the command line.

- allow me to return from anywhere in a thread wthout getting a message to STDERR (anyone objects to stealing exit() for this?)\, which I'll probably call Thread​::Exit.

- lot's of other nice stuff involving events.

Liz

-- H.Merijn Brand Amsterdam Perl Mongers (http​://amsterdam.pm.org/) using perl-5.6.1\, 5.8.0 & 633 on HP-UX 10.20 & 11.00\, AIX 4.2\, AIX 4.3\,   WinNT 4\, Win2K pro & WinCE 2.11. Smoking perl CORE​: smokers@​perl.org http​://archives.develooper.com/daily-build@​perl.org/ perl-qa@​perl.org send smoke reports to​: smokers-reports@​perl.org\, QA​: http​://qa.perl.org

[p5pRT]

From @lizmat

At 08​:45 AM 9/4/02 +0200\, H.Merijn Brand wrote​:

But what I was thinking\, was more along the lines of​: And where do you use $new now?

Oops. I should have known better than to write this down from memory after a 2 hour drive in the middle of the night. Now that my development machine is up\, I can cut and past what I thought of earlier yesterday​:

  my $new = \&threads​::new;   *threads​::new = sub {   my $class = shift;   my $sub = shift;   $new->( $class\,sub {   * do what I want to do inside the thread here   goto &$sub;   }\,@​_ );

Liz

[p5pRT]

From nick.ing-simmons@elixent.com

Elizabeth Mattijsen \liz@​dijkmat\.nl writes​:

At 06​:25 PM 9/2/02 -0400\, Benjamin Goldberg wrote​:

Would it be possible to get at the new thread id and the getpid() of the new thread in XS from within a CLONE subroutine? Within the CLONE method\, the "current thread id" should *be* the "new thread id". Regardless of whether or not doing it in XS.

Well\, I agree. But that is _not_ the case right now.

And the new OS-thread may not yet have been created.

Well\, after some more thinking\, I created a work around that seems to work. By hijacking threads​::new\, you can run a subroutine inside the thread before the actual thread subroutine is started.

It might make sense for threads​::new to always call threads​::something before calling user's sub.

Why wasn't CLONE implemented that way?

As I tried to explain before - if child thread is to get a true "clone" at instant it was created (and indeed to avoid races on these un-shared things) the parent thread must be suspended and not modifying things while it happens\, and then re-awoken.

The current CLONE fakes this by having the parent thread do the clone (so it isn't doing anything else).

-- Nick Ing-Simmons http​://www.ni-s.u-net.com/

[p5pRT]

From @lizmat

At 04​:13 PM 9/5/02 +0100\, Nick Ing-Simmons wrote​:

Well\, after some more thinking\, I created a work around that seems to work. By hijacking threads​::new\, you can run a subroutine inside the thread before the actual thread subroutine is started. It might make sense for threads​::new to always call threads​::something before calling user's sub.

Hmmm... I could add a begin() method to Thread​::Exit\, analogous to the end() class method. Writing the documentation would be more work.

Why wasn't CLONE implemented that way? The current CLONE fakes this by having the parent thread do the clone (so it isn't doing anything else).

Now that makes sense to me.

Liz

[p5pRT]

From arthur@contiller.se

Hi\, This is now solved.

We do this by faking what thread we are running in during destroy and clone operations so the tid is correctly reported.

However you will not get a correct underlying OS tid\, nor a correct pid in linux\, however one should not depend on the underlying OS tid\, neither should one look at the pid since we are free to migrate perl threads across OS level threads.

The patch that fixes this is #18417

Arthur

[p5pRT]

arthur@contiller.se - Status changed from 'open' to 'resolved'