Perl / perl5

🐪 The Perl programming language
https://dev.perl.org/perl5/
Other
1.91k stars 542 forks source link

proposed fix for references to tied hash keys #7271

Open p5pRT opened 20 years ago

p5pRT commented 20 years ago

Migrated from rt.perl.org#29224 (status was 'open')

Searchable as RT29224$

p5pRT commented 20 years ago

From @muir

Created by @muir

I've run into many bugs with references to tied hash keys. Some of them are documented in bug # 27555. In addition to what's there\, I've had core dumps that I haven't reported because I couldn't find a self-contained test case. Basically I've concluded that refereces to tied hash keys are too broken to use.

Even if all of the bugs in 27555 were fixed (and the core dumps too)\, there would still be a problem​: tied hash code sometimes needs to know that a reference has been made. My soon-to-be-released persistent object store module uses tied hashes that very much need to know about references.

I propose that an additional tie method​: GETREF.

GETREF would be invoked whenver someone did​:

  $x = \%tiedhash{key}

It would return the reference.

If a GETREF method isn't provided\, perl can fall back to the current broken behavior.

An example from my module... Values in $hash1 and $hash2 are only loaded into memory as they are accessed.

  $hash1->{x} = \$hash2->{y};

  ... time passes and $hash1 is saved persistently and is   no longer in memory...

  $hash1->{z} = \$hash2->{y}; # a ref is made   delete $hash2->{y};
  ${$hash1->{z}} = 'a'; # change the value   $hash1->{b} = ${$hash1->{x}}; # $hash1->{x} is loaded into memory

in my module $hash1->{b} is 'a' but that's only possible because I fake up a call to GETREF. Just fixing the bug # 27555 would not allow this to work.

Here's how I'm working around the lack of GETREF​:

  use B qw(svref_2object);

  sub workaround27555   {   my $self = shift;   my ($tiedaddr\, $key) = tied_hash_reference($_[0]);   print "workaround27555($qaddr{\$_[0]}) no tied addr\n" if $debug_27555 && ! $tiedaddr;   return $_[0] unless $tiedaddr;

  ... application specific code to translate from $tiedaddr to $tied ...

  $_[0] = $tied->GETREF($key);   print "workaround27555($qaddr{\$_[0]}) references %*$id/'$key - replaced with GETREF\n" if $debug_27555;   return $_[0];   }

  sub tied_hash_reference   {   my ($ref) = @​_;   local($@​);   return eval {   my $magic = svref_2object($ref)->MAGIC;   $magic = $magic->MOREMAGIC   while lc($magic->TYPE) ne 'p';   return (${$magic->OBJ->RV}\, $magic->PTR->as_string);   };   }

It's used like​:  
  $x = \%tiedhash{key}   $myobj->workaround27555($x);

I also have an explicit getref function alternative​:

  use UNIVERSAL qw(can);

  sub getref(\%$)   {   my $hash = shift;   my $key = shift;   my $tied = tied %$hash;   die unless reftype($hash) eq 'HASH';   return \$hash->{$key} unless $tied && $tied->can('GETREF');   print "getref getting references for '$key'\n" if $debug_27555;   return $tied->GETREF($key);   }

It's used as​:

  $x = getref(%tiedhash\, 'key');

Perl Info ``` Flags: category=core severity=wishlist Site configuration information for perl v5.8.3: Configured by Debian Project at Sat Mar 27 17:07:14 EST 2004. Summary of my perl5 (revision 5.0 version 8 subversion 3) configuration: Platform: osname=linux, osvers=2.4.25-ti1211, archname=i386-linux-thread-multi uname='linux kosh 2.4.25-ti1211 #1 thu feb 19 18:20:12 est 2004 i686 gnulinux ' config_args='-Dusethreads -Duselargefiles -Dccflags=-DDEBIAN -Dcccdlflags=-fPIC -Darchname=i386-linux -Dprefix=/usr -Dprivlib=/usr/share/perl/5.8 -Darchlib=/usr/lib/perl/5.8 -Dvendorprefix=/usr -Dvendorlib=/usr/share/perl5 -Dvendorarch=/usr/lib/perl5 -Dsiteprefix=/usr/local -Dsitelib=/usr/local/share/perl/5.8.3 -Dsitearch=/usr/local/lib/perl/5.8.3 -Dman1dir=/usr/share/man/man1 -Dman3dir=/usr/share/man/man3 -Dsiteman1dir=/usr/local/man/man1 -Dsiteman3dir=/usr/local/man/man3 -Dman1ext=1 -Dman3ext=3perl -Dpager=/usr/bin/sensible-pager -Uafs -Ud_csh -Uusesfio -Uusenm -Duseshrplib -Dlibperl=libperl.so.5.8.3 -Dd_dosuid -des' hint=recommended, useposix=true, d_sigaction=define usethreads=define use5005threads=undef useithreads=define usemultiplicity=define useperlio=define d_sfio=undef uselargefiles=define usesocks=undef use64bitint=undef use64bitall=undef uselongdouble=undef usemymalloc=n, bincompat5005=undef Compiler: cc='cc', ccflags ='-D_REENTRANT -D_GNU_SOURCE -DTHREADS_HAVE_PIDS -DDEBIAN -fno-strict-aliasing -I/usr/local/include -D_LARGEFILE_SOURCE -D_FILE_OFFSET_BITS=64', optimize='-O3', cppflags='-D_REENTRANT -D_GNU_SOURCE -DTHREADS_HAVE_PIDS -DDEBIAN -fno-strict-aliasing -I/usr/local/include' ccversion='', gccversion='3.3.3 (Debian 20040314)', gccosandvers='' intsize=4, longsize=4, ptrsize=4, doublesize=8, byteorder=1234 d_longlong=define, longlongsize=8, d_longdbl=define, longdblsize=12 ivtype='long', ivsize=4, nvtype='double', nvsize=8, Off_t='off_t', lseeksize=8 alignbytes=4, prototype=define Linker and Libraries: ld='cc', ldflags =' -L/usr/local/lib' libpth=/usr/local/lib /lib /usr/lib libs=-lgdbm -lgdbm_compat -ldb -ldl -lm -lpthread -lc -lcrypt perllibs=-ldl -lm -lpthread -lc -lcrypt libc=/lib/libc-2.3.2.so, so=so, useshrplib=true, libperl=libperl.so.5.8.3 gnulibc_version='2.3.2' Dynamic Linking: dlsrc=dl_dlopen.xs, dlext=so, d_dlsymun=undef, ccdlflags='-rdynamic' cccdlflags='-fPIC', lddlflags='-shared -L/usr/local/lib' Locally applied patches: @INC for perl v5.8.3: /etc/perl /usr/local/lib/perl/5.8.3 /usr/local/share/perl/5.8.3 /usr/lib/perl5 /usr/share/perl5 /usr/lib/perl/5.8 /usr/share/perl/5.8 /usr/local/lib/site_perl /usr/local/lib/perl/5.8.2 /usr/local/share/perl/5.8.2 . Environment for perl v5.8.3: HOME=/home/muir LANG=C LANGUAGE (unset) LD_LIBRARY_PATH (unset) LOGDIR (unset) PATH=.:/home/muir/bin/charm:/home/muir/bin:/home/muir/bin/share:/bin:/usr/bin:/sbin:/usr/sbin:/usr/local/shbin:/usr/local/sbin:/usr/local/bin:/usr/local/ptybin:/usr/X11R6/bin:/usr/bin/X11:/usr/local/tex/bin:/usr/ucb:/usr/bin:/bin:/etc:/usr/etc:/usr/games:/lib:/usr/lib:/usr/local/java/bin:/usr/lib/uucp:/usr/openwin/bin:/usr/openwin/bin/xview:/usr/openwin/demo:/usr/adm:/home/muir/tmp PERL_BADLANG (unset) SHELL=/bin/tcsh ```
p5pRT commented 11 years ago

From @jkeenan

On Wed Apr 28 00​:57​:52 2004\, muir@​idiom.com wrote​:

This is a bug report for perl from muir@​idiom.com\, generated with the help of perlbug 1.34 running under perl v5.8.3.

----------------------------------------------------------------- [Please enter your report here]

I've run into many bugs with references to tied hash keys. Some of them are documented in bug # 27555. In addition to what's there\, I've had core dumps that I haven't reported because I couldn't find a self-contained test case. Basically I've concluded that refereces to tied hash keys are too broken to use.

Even if all of the bugs in 27555 were fixed (and the core dumps too)\, there would still be a problem​: tied hash code sometimes needs to know that a reference has been made. My soon-to-be-released persistent object store module uses tied hashes that very much need to know about references.

Was that module ever released to CPAN? (Nothing at http​://search.cpan.org/~muir/ leaps out to me.)

I propose that an additional tie method​: GETREF.

GETREF would be invoked whenver someone did​:

$x = \\%tiedhash\{key\}

It would return the reference.

If a GETREF method isn't provided\, perl can fall back to the current broken behavior.

An example from my module... Values in $hash1 and $hash2 are only loaded into memory as they are accessed.

$hash1\->\{x\} = \\$hash2\->\{y\};

\.\.\. time passes and $hash1 is saved persistently and is
no longer in memory\.\.\.

$hash1\->\{z\} = \\$hash2\->\{y\};        \# a ref is made
delete $hash2\->\{y\};
$\{$hash1\->\{z\}\} = 'a';            \# change the value
$hash1\->\{b\} = $\{$hash1\->\{x\}\};        \# $hash1\->\{x\} is loaded into memory

in my module $hash1->{b} is 'a' but that's only possible because I fake up a call to GETREF. Just fixing the bug # 27555 would not allow this to work.

Here's how I'm working around the lack of GETREF​:

use B qw\(svref\_2object\);

sub workaround27555
\{
    my $self = shift;
    my \($tiedaddr\, $key\) = tied\_hash\_reference\($\_\[0\]\);
    print "workaround27555\($qaddr\{\\$\_\[0\]\}\) no tied addr\\n" if

$debug_27555 && ! $tiedaddr; return $_[0] unless $tiedaddr;

    \.\.\. application specific code to translate from $tiedaddr to $tied

...

    $\_\[0\] = $tied\->GETREF\($key\);
    print "workaround27555\($qaddr\{\\$\_\[0\]\}\) references %\*$id/'$key \-

replaced with GETREF\n" if $debug_27555; return $_[0]; }

sub tied\_hash\_reference
\{
    my \($ref\) = @​\_;
    local\($@​\);
    return eval \{
        my $magic = svref\_2object\($ref\)\->MAGIC;
        $magic = $magic\->MOREMAGIC
            while lc\($magic\->TYPE\) ne 'p';
        return \($\{$magic\->OBJ\->RV\}\, $magic\->PTR\->as\_string\);
    \};
\}

It's used like​:

$x = \\%tiedhash\{key\}
$myobj\->workaround27555\($x\);

I also have an explicit getref function alternative​:

use UNIVERSAL qw\(can\);

sub getref\(\\%$\)
\{
    my $hash = shift;
    my $key = shift;
    my $tied = tied %$hash;
    die unless reftype\($hash\) eq 'HASH';
    return \\$hash\->\{$key\} unless $tied && $tied\->can\('GETREF'\);
    print "getref getting references for '$key'\\n" if $debug\_27555;
    return $tied\->GETREF\($key\);
\}

It's used as​:

$x = getref\(%tiedhash\, 'key'\);

Thank you very much. Jim Keenan

p5pRT commented 11 years ago

The RT System itself - Status changed from 'new' to 'open'

p5pRT commented 11 years ago

From @muir

Yes. It is OOPS (http​://search.cpan.org/~muir/OOPS/).

I developed a workaround for the issue that complicates use of my module (as a user of the module\, you have to do something special with your tied hash key references so that OOPS can handle them). My workaround uses B​::svref_2object.

I would still like the GETREF tie method.

Thanks\, -Dave

On Thu\, Oct 11\, 2012 at 5​:21 PM\, James E Keenan via RT \< perlbug-followup@​perl.org> wrote​:

On Wed Apr 28 00​:57​:52 2004\, muir@​idiom.com wrote​:

This is a bug report for perl from muir@​idiom.com\, generated with the help of perlbug 1.34 running under perl v5.8.3.

----------------------------------------------------------------- [Please enter your report here]

I've run into many bugs with references to tied hash keys. Some of them are documented in bug # 27555. In addition to what's there\, I've had core dumps that I haven't reported because I couldn't find a self-contained test case. Basically I've concluded that refereces to tied hash keys are too broken to use.

Even if all of the bugs in 27555 were fixed (and the core dumps too)\, there would still be a problem​: tied hash code sometimes needs to know that a reference has been made. My soon-to-be-released persistent object store module uses tied hashes that very much need to know about references.

Was that module ever released to CPAN? (Nothing at http​://search.cpan.org/~muir/ leaps out to me.)

I propose that an additional tie method​: GETREF.

GETREF would be invoked whenver someone did​:

  $x = \\%tiedhash\{key\}

It would return the reference.

If a GETREF method isn't provided\, perl can fall back to the current broken behavior.

An example from my module... Values in $hash1 and $hash2 are only loaded into memory as they are accessed.

  $hash1\->\{x\} = \\$hash2\->\{y\};

  \.\.\. time passes and $hash1 is saved persistently and is
  no longer in memory\.\.\.

  $hash1\->\{z\} = \\$hash2\->\{y\};             \# a ref is made
  delete $hash2\->\{y\};
  $\{$hash1\->\{z\}\} = 'a';                   \# change the value
  $hash1\->\{b\} = $\{$hash1\->\{x\}\};           \# $hash1\->\{x\} is loaded

into memory

in my module $hash1->{b} is 'a' but that's only possible because I fake up a call to GETREF. Just fixing the bug # 27555 would not allow this to work.

Here's how I'm working around the lack of GETREF​:

  use B qw\(svref\_2object\);

  sub workaround27555
  \{
          my $self = shift;
          my \($tiedaddr\, $key\) = tied\_hash\_reference\($\_\[0\]\);
          print "workaround27555\($qaddr\{\\$\_\[0\]\}\) no tied addr\\n" if

$debug_27555 && ! $tiedaddr; return $_[0] unless $tiedaddr;

          \.\.\. application specific code to translate from $tiedaddr

to $tied ...

          $\_\[0\] = $tied\->GETREF\($key\);
          print "workaround27555\($qaddr\{\\$\_\[0\]\}\) references

%*$id/'$key - replaced with GETREF\n" if $debug_27555; return $_[0]; }

  sub tied\_hash\_reference
  \{
          my \($ref\) = @&#8203;\_;
          local\($@&#8203;\);
          return eval \{
                  my $magic = svref\_2object\($ref\)\->MAGIC;
                  $magic = $magic\->MOREMAGIC
                          while lc\($magic\->TYPE\) ne 'p';
                  return \($\{$magic\->OBJ\->RV\}\,

$magic->PTR->as_string); }; }

It's used like​:

  $x = \\%tiedhash\{key\}
  $myobj\->workaround27555\($x\);

I also have an explicit getref function alternative​:

  use UNIVERSAL qw\(can\);

  sub getref\(\\%$\)
  \{
          my $hash = shift;
          my $key = shift;
          my $tied = tied %$hash;
          die unless reftype\($hash\) eq 'HASH';
          return \\$hash\->\{$key\} unless $tied && $tied\->can\('GETREF'\);
          print "getref getting references for '$key'\\n" if

$debug_27555; return $tied->GETREF($key); }

It's used as​:

  $x = getref\(%tiedhash\, 'key'\);

Thank you very much. Jim Keenan

--- via perlbug​: queue​: perl5 status​: new https://rt-archive.perl.org/perl5/Ticket/Display.html?id=29224

p5pRT commented 8 years ago

From @dcollinsn

Is there anything blocking this request to add the GETREF tie method?

p5pRT commented 8 years ago

From @muir

On Wed Jul 06 13​:17​:48 2016\, dcollinsn@​gmail.com wrote​:

Is there anything blocking this request to add the GETREF tie method?

Not that I know of.

p5pRT commented 8 years ago

From [Unknown Contact. See original ticket]

On Wed Jul 06 13​:17​:48 2016\, dcollinsn@​gmail.com wrote​:

Is there anything blocking this request to add the GETREF tie method?

Not that I know of.