Perl / perl5

🐪 The Perl programming language
https://dev.perl.org/perl5/
Other
1.96k stars 556 forks source link

Threads, shared memory, blessed objects fail to call ->DESTROY method #15839

Open p5pRT opened 7 years ago

p5pRT commented 7 years ago

Migrated from rt.perl.org#130649 (status was 'open')

Searchable as RT130649$

p5pRT commented 7 years ago

From victor.burns@bankofamerica.com

This issue is believed to exist in all 5.10.1+ Perl's and every older Perl as well. I have verified the issue in every Perl I have at my disposal from 5.10.1\, 5.12.0\, 5.22.0 and 5.24.0.

Description​: If the last reference to an object is in thread shared memory the objects DESTROY method won't be called or highly unlikely to be called. This is a *huge* problem for applications that are OO and designed using composition of other classes (HAS-A). The last\, and possibly only reference will be in Perl's thread shared memory. I think the real issue only resembles my description and therefore it is not perfect. I have however included a small script that when used and modified as the comments suggest the related behaviors may be observed. The actual issue is a bit more unpredictable.

## PERL BUG Example

use 5.010001; use strict; use warnings; use threads; use threads​::shared;

our $VERSION = 0.1;

say 'Hello World!\, - Perl shared memory failure to call ->DESTROY detector.';

LEAKER​: {   ## Using the "shared" attribute can play a role in this   ##  Best guess it means all references are shared.   my $aa :shared = bless &share({})\, 'Lazy​::Crazy';   my $bb :shared = bless &share({})\, 'Lazy​::Crazy';   my $cc :shared = &share( bless {}\, 'Lazy​::Crazy' );

  ## Placing an object into another makes it a sure bet its   ##  ->DESTROY won't be called   $aa->{ has_c } = $cc;   $bb->{ has_a } = $aa;

  ## It's strange that the order of the following assignments   ##  will allow the last one touched to call its ->DESTROY   $cc->{ label } = 'c';   $aa->{ label } = 'a';   $bb->{ label } = 'b';

  ## If I'm able to control the order these are unwrapped (destroyed)   ##  things start working as expected.   ##    How likely can I always get this right\, not at all!   ## (uncomment and things work as expected) # $bb = undef; # $aa = undef; # $cc = undef; }

package Lazy​::Crazy;

sub DESTROY {   my $self = shift;   printf "# %s '%s' is destroyed\n"\, ref $self\, $self->{ label };   return; }

## END

Perl Info ``` Flags:     category=library     severity=high     module=threads::shared Site configuration information for perl 5.12.2: Configured by SYSTEM at Mon Sep  6 23:12:49 2010. Summary of my perl5 (revision 5 version 12 subversion 2) configuration:      Platform:     osname=MSWin32, osvers=5.00, archname=MSWin32-x86-multi-thread     uname=''     config_args='undef'     hint=recommended, useposix=true, d_sigaction=undef     useithreads=define, usemultiplicity=define     useperlio=define, d_sfio=undef, uselargefiles=define, usesocks=undef     use64bitint=undef, use64bitall=undef, uselongdouble=undef     usemymalloc=n, bincompat5005=undef   Compiler:     cc='cl', ccflags ='-nologo -GF -W3 -MD -Zi -DNDEBUG -O1 -DWIN32 -D_CONSOLE -DNO_STRICT -DHAVE_DES_FCRYPT -DUSE_SITECUSTOMIZE -DPERL_IMPLICIT_CONTEXT -DPERL_IMPLICIT_SYS -DUSE_PERLIO -D_USE_32BIT_TIME_T -DPERL_MSVCRT_READFIX',     optimize='-MD -Zi -DNDEBUG -O1',     cppflags='-DWIN32'     ccversion='12.00.8804', gccversion='', gccosandvers=''     intsize=4, longsize=4, ptrsize=4, doublesize=8, byteorder=1234     d_longlong=undef, longlongsize=8, d_longdbl=define, longdblsize=8     ivtype='long', ivsize=4, nvtype='double', nvsize=8, Off_t='__int64', lseeksize=8     alignbytes=8, prototype=define   Linker and Libraries:     ld='link', ldflags ='-nologo -nodefaultlib -debug -opt:ref,icf  -libpath:"C:\Program Files (x86)\Perl\lib\CORE"  -machine:x86'     libpth=\lib     libs=  oldnames.lib kernel32.lib user32.lib gdi32.lib winspool.lib  comdlg32.lib advapi32.lib shell32.lib ole32.lib oleaut32.lib  netapi32.lib uuid.lib ws2_32.lib mpr.lib winmm.lib  version.lib odbc32.lib odbccp32.lib comctl32.lib msvcrt.lib     perllibs=  oldnames.lib kernel32.lib user32.lib gdi32.lib winspool.lib  comdlg32.lib advapi32.lib shell32.lib ole32.lib oleaut32.lib  netapi32.lib uuid.lib ws2_32.lib mpr.lib winmm.lib  version.lib odbc32.lib odbccp32.lib comctl32.lib msvcrt.lib     libc=msvcrt.lib, so=dll, useshrplib=true, libperl=perl512.lib     gnulibc_version=''   Dynamic Linking:     dlsrc=dl_win32.xs, dlext=dll, d_dlsymun=undef, ccdlflags=' '     cccdlflags=' ', lddlflags='-dll -nologo -nodefaultlib -debug -opt:ref,icf  -libpath:"C:\Program Files (x86)\Perl\lib\CORE"  -machine:x86' Locally applied patches:     ACTIVEPERL_LOCAL_PATCHES_ENTRY     1fd8fa4 Add Wolfram Humann to AUTHORS     f120055 make string-append on win32 100 times faster     a2a8d15 Define _USE_32BIT_TIME_T for VC6 and VC7     007cfe1 Don't pretend to support really old VC++ compilers     6d8f7c9 Get rid of obsolete PerlCRT.dll support     d956618 Make Term::ReadLine::findConsole fall back to STDIN if /dev/tty can't be opened     321e50c Escape patch strings before embedding them in patchlevel.h @INC for perl 5.12.2:     C:/Program Files (x86)/Perl/site/lib     C:/Program Files (x86)/Perl/lib     . Environment for perl 5.12.2:     HOME (unset)     LANG (unset)     LANGUAGE (unset)     LD_LIBRARY_PATH (unset)     LOGDIR (unset)     PATH=C:\ProgramData\Oracle\Java\javapath;C:\Oracle\product\11.2.0\client_1\bin;C:\Program Files (x86)\Perl\site\bin;C:\Program Files (x86)\Perl\bin;C:\windows\system32;C:\windows;C:\windows\System32\Wbem;C:\windows\System32\WindowsPowerShell\v1.0\;C:\Program Files (x86)\PKWARE\pkzipc;C:\Program Files (x86)\Microsoft Application Virtualization Client;C:\Program Files (x86)\SSH Communications Security\SSH Tectia\SSH Tectia AUX;C:\Program Files (x86)\SSH Communications Security\SSH Tectia\SSH Tectia AUX\Support binaries;C:\Program Files (x86)\SSH Communications Security\SSH Tectia\SSH Tectia Broker;C:\Program Files (x86)\SSH Communications Security\SSH Tectia\SSH Tectia Client;C:\windows\System32\WindowsPowerShell\v1.0\;C:\Program Files\1E\NomadBranch\     PERL_BADLANG (unset)     SHELL (unset) ---------------------------------------------------------------------- This message, and any attachments, is for the intended recipient(s) only, may contain information that is privileged, confidential and/or proprietary and subject to important terms and conditions available at http://www.bankofamerica.com/emaildisclaimer. If you are not the intended recipient, please delete this message. ```
p5pRT commented 7 years ago

From @jkeenan

On Thu\, 26 Jan 2017 12​:38​:49 GMT\, victor.burns@​bankofamerica.com wrote​:

This is a bug report for Perl from victor.burns@​bankofamerica.com\, generated with the help of perlbug 1.39 running under perl 5.12.2.

----------------------------------------------------------------- [Please describe your issue here]

This issue is believed to exist in all 5.10.1+ Perl's and every older Perl as well. I have verified the issue in every Perl I have at my disposal from 5.10.1\, 5.12.0\, 5.22.0 and 5.24.0.

Description​: If the last reference to an object is in thread shared memory the objects DESTROY method won't be called or highly unlikely to be called. This is a *huge* problem for applications that are OO and designed using composition of other classes (HAS-A). The last\, and possibly only reference will be in Perl's thread shared memory. I think the real issue only resembles my description and therefore it is not perfect. I have however included a small script that when used and modified as the comments suggest the related behaviors may be observed. The actual issue is a bit more unpredictable.

## PERL BUG Example

use 5.010001; use strict; use warnings; use threads; use threads​::shared;

our $VERSION = 0.1;

say 'Hello World!\, - Perl shared memory failure to call ->DESTROY detector.';

LEAKER​: {   ## Using the "shared" attribute can play a role in this   ##  Best guess it means all references are shared.   my $aa :shared = bless &share({})\, 'Lazy​::Crazy';   my $bb :shared = bless &share({})\, 'Lazy​::Crazy';   my $cc :shared = &share( bless {}\, 'Lazy​::Crazy' );

  ## Placing an object into another makes it a sure bet its   ##  ->DESTROY won't be called   $aa->{ has_c } = $cc;   $bb->{ has_a } = $aa;

  ## It's strange that the order of the following assignments   ##  will allow the last one touched to call its ->DESTROY   $cc->{ label } = 'c';   $aa->{ label } = 'a';   $bb->{ label } = 'b';

  ## If I'm able to control the order these are unwrapped (destroyed)   ##  things start working as expected.   ##    How likely can I always get this right\, not at all!   ## (uncomment and things work as expected) # $bb = undef; # $aa = undef; # $cc = undef; }

package Lazy​::Crazy;

sub DESTROY {   my $self = shift;   printf "# %s '%s' is destroyed\n"\, ref $self\, $self->{ label };   return; }

## END

I would like to confirm the output one is supposed to get from running this program.

Running the program as submitted (which\, for convenience\, I am attaching to this RT)\, I get​:

##### [threaded_blead] $ ./bin/perl -Ilib ~/learn/perl/p5p/130649-threads.pl Hello World!\, - Perl shared memory failure to call ->DESTROY detector. # Lazy​::Crazy 'b' is destroyed #####

Uncommenting lines 34-36 -- the last 3 lines of the 'LEAKER' block -- and then running the program I get​:

##### [threaded_blead] $ ./bin/perl -Ilib ~/learn/perl/p5p/130649-threads.pl Hello World!\, - Perl shared memory failure to call ->DESTROY detector. # Lazy​::Crazy 'b' is destroyed # Lazy​::Crazy 'a' is destroyed # Lazy​::Crazy 'c' is destroyed #####

Is that the output you were expecting?

Thank you very much.

-- James E Keenan (jkeenan@​cpan.org)

p5pRT commented 7 years ago

From @jkeenan

130649-threads.pl

p5pRT commented 7 years ago

The RT System itself - Status changed from 'new' to 'open'

p5pRT commented 7 years ago

From victor.burns@bankofamerica.com

Yes\, when it works properly each of the constructed objects should announce its demise when the ->DESTROY method is called. When script is run as given OR modified in some of the other suggested ways the results will vary including the possibility that none of the objects will announce their demise.

Best Regards\,

Victor Burns | TG Application Architect / Developer & Perl GURU AVP\, Consultant II System Engineer\, Bank of America T 469.201.8375 | victor.burns@​bankofamerica.com 16001 N Dallas Pkwy\, Addison\, TX 75001

  Like us on Facebook   Follow us on Twitter

Life’s better when we’re connected™

-----Original Message----- From​: James E Keenan via RT [mailto​:perlbug-followup@​perl.org] Sent​: Thursday\, January 26\, 2017 8​:28 AM To​: Burns\, Victor M \victor\.burns@​bankofamerica\.com Subject​: [perl #130649] Threads\, shared memory\, blessed objects fail to call ->DESTROY method

I would like to confirm the output one is supposed to get from running this program.

Running the program as submitted (which\, for convenience\, I am attaching to this RT)\, I get​:

##### [threaded_blead] $ ./bin/perl -Ilib ~/learn/perl/p5p/130649-threads.pl Hello World!\, - Perl shared memory failure to call ->DESTROY detector. # Lazy​::Crazy 'b' is destroyed #####

Uncommenting lines 34-36 -- the last 3 lines of the 'LEAKER' block -- and then running the program I get​:

##### [threaded_blead] $ ./bin/perl -Ilib ~/learn/perl/p5p/130649-threads.pl Hello World!\, - Perl shared memory failure to call ->DESTROY detector. # Lazy​::Crazy 'b' is destroyed # Lazy​::Crazy 'a' is destroyed # Lazy​::Crazy 'c' is destroyed #####

Is that the output you were expecting?

Thank you very much.

-- James E Keenan (jkeenan@​cpan.org)


This message\, and any attachments\, is for the intended recipient(s) only\, may contain information that is privileged\, confidential and/or proprietary and subject to important terms and conditions available at http​://www.bankofamerica.com/emaildisclaimer. If you are not the intended recipient\, please delete this message.

p5pRT commented 7 years ago

From @jdhedden

The bug is that the destructor of an embedded shared object will not be called after the enclosing shared object is destroyed​:

1. Create shared objects $xx and $yy. 2. Embed $yy inside $XX​: $xx->{embedded} = $yy 3. Undef $yy. Its destructor is not called because it's still referenced by $xx. (This is as expected.) 4. Undef $xx. The destructor for $xx is called (as expected)\, but the destructor for $yy is not (bug).

If the above is done with ordinary objects\, the the destructors for both $xx and $yy will be called\, as expected.

One workaround is to code the destructor to actively dereference any embedded objects.

The attached code illustrates both the bug and the workaround.

p5pRT commented 7 years ago

From @jdhedden

ts_embed_destory.pl

p5pRT commented 7 years ago

From [Unknown Contact. See original ticket]

The bug is that the destructor of an embedded shared object will not be called after the enclosing shared object is destroyed​:

1. Create shared objects $xx and $yy. 2. Embed $yy inside $XX​: $xx->{embedded} = $yy 3. Undef $yy. Its destructor is not called because it's still referenced by $xx. (This is as expected.) 4. Undef $xx. The destructor for $xx is called (as expected)\, but the destructor for $yy is not (bug).

If the above is done with ordinary objects\, the the destructors for both $xx and $yy will be called\, as expected.

One workaround is to code the destructor to actively dereference any embedded objects.

The attached code illustrates both the bug and the workaround.