push on scalar forbidden: why?

[p5pRT]

Migrated from rt.perl.org#133109 (status was 'open')

Searchable as RT133109$

[p5pRT]

From perl-diddler@tlinx.org

Created by perl-diddler@tlinx.org

If a scaler is null or an ARRAY ref\, why shouldn't perl be able to treat it as one in a context where a non-ARRAY ref would generate an error?

ex​:   my $aref;   push $aref\, 1;

I'm aware an alternate way to do this was added\, but perl\, like many languages often has multiple ways to do something. More so in perl in that a major feature in perl is autovivification.

This is a perfect example of perl autovivifying an array ref to use as an array needed by push. So why would it be "forbidden".

Even more curious is that perl is a permissive language. If you want to do something it will usually let you -- assuming that the user is in a position to better know what is wanted/needed in a specific circumstance. I remember a discussion of perl's OO design and how perl doesn't enforce class boundaries or the concept of 'private'\, by _default_. While extensions can can add such things\, nothing is forbidden. So why would something that used to work be made "forbidden"?

Can perl be allowed to do the right thing when perl already knows what it is?

Thanks\,

Perl Info

``` Flags: category=core severity=high Site configuration information for perl 5.26.1: Configured by ASSI at Tue Sep 26 18:50:45 CEST 2017. Summary of my perl5 (revision 5 version 26 subversion 1) configuration: Platform: osname=cygwin osvers=2.9.0(0.31853) archname=x86_64-cygwin-threads-multi uname='cygwin_nt-6.3 cygwin 2.9.0(0.31853) 2017-09-12 10:18 x86_64 cygwin ' config_args='-des -Dprefix=/usr -Dmksymlinks -Darchname=x86_64-cygwin-threads -Dlibperl=cygperl5_26.dll -Dcc=gcc -Dld=g++ -Accflags=-ggdb -O2 -pipe -Wimplicit-function-declaration -fdebug-prefix-map=/mnt/share/maint/perl.x86_64/build=/usr/src/debug/perl-5.26.1-1 -fdebug-prefix-map=/mnt/share/maint/perl.x86_64/src/perl-5.26.1=/usr/src/debug/perl-5.26.1-1 -fwrapv' hint=recommended useposix=true d_sigaction=define useithreads=define usemultiplicity=define use64bitint=define use64bitall=define uselongdouble=undef usemymalloc=n default_inc_excludes_dot=define bincompat5005=undef Compiler: cc='gcc' ccflags ='-DPERL_USE_SAFE_PUTENV -U__STRICT_ANSI__ -D_GNU_SOURCE -ggdb -O2 -pipe -Wimplicit-function-declaration -fdebug-prefix-map=/mnt/share/maint/perl.x86_64/build=/usr/src/debug/perl-5.26.1-1 -fdebug-prefix-map=/mnt/share/maint/perl.x86_64/src/perl-5.26.1=/usr/src/debug/perl-5.26.1-1 -fwrapv -fno-strict-aliasing -fstack-protector-strong -D_FORTIFY_SOURCE=2' optimize='-O3' cppflags='-DPERL_USE_SAFE_PUTENV -U__STRICT_ANSI__ -D_GNU_SOURCE -ggdb -O2 -pipe -Wimplicit-function-declaration -fdebug-prefix-map=/mnt/share/maint/perl.x86_64/build=/usr/src/debug/perl-5.26.1-1 -fdebug-prefix-map=/mnt/share/maint/perl.x86_64/src/perl-5.26.1=/usr/src/debug/perl-5.26.1-1 -fwrapv -fno-strict-aliasing -fstack-protector-strong' ccversion='' gccversion='6.4.0' gccosandvers='' intsize=4 longsize=8 ptrsize=8 doublesize=8 byteorder=12345678 doublekind=3 d_longlong=define longlongsize=8 d_longdbl=define longdblsize=16 longdblkind=3 ivtype='long' ivsize=8 nvtype='double' nvsize=8 Off_t='off_t' lseeksize=8 alignbytes=8 prototype=define Linker and Libraries: ld='g++' ldflags =' -Wl,--enable-auto-import -Wl,--export-all-symbols -Wl,--enable-auto-image-base -fstack-protector-strong' libpth=/usr/lib libs=-lpthread -lgdbm -ldb -ldl -lcrypt -lgdbm_compat perllibs=-lpthread -ldl -lcrypt libc=/usr/lib/libcygwin.a so=dll useshrplib=true libperl=cygperl5_26.dll gnulibc_version='' Dynamic Linking: dlsrc=dl_dlopen.xs dlext=dll d_dlsymun=undef ccdlflags=' ' cccdlflags=' ' lddlflags=' --shared -Wl,--enable-auto-import -Wl,--export-all-symbols -Wl,--enable-auto-image-base -fstack-protector-strong' Locally applied patches: Cygwin: README Cygwin: use auto-image-base instead of fixed DLL base address Cygwin: modify hints Cygwin: Configure correct libsearch Cygwin: Configure correct libpth Cygwin: Win32 correct UTF8 handling Perl: File-Path-2.14 (fixes CVE2017-6512) @INC for perl 5.26.1: /usr/local/lib/perl5/site_perl/5.26/x86_64-cygwin-threads /usr/local/share/perl5/site_perl/5.26 /usr/lib/perl5/vendor_perl/5.26/x86_64-cygwin-threads /usr/share/perl5/vendor_perl/5.26 /usr/lib/perl5/5.26/x86_64-cygwin-threads /usr/share/perl5/5.26 Environment for perl 5.26.1: CYGWIN=system nodosfilewarning winsymlinks:native export HOME=/Users/l.B LANG (unset) LANGUAGE (unset) LC_COLLATE=C LC_CTYPE=en_US.UTF-8 LC_MESSAGES=C LC_MONETARY=C LC_NUMERIC=C LC_TIME=C LD_LIBRARY_PATH (unset) LOGDIR (unset) PATH=.:.:/usr/bin:/sbin:/prog64/vim/current:/Prog64/VanDyke Software/SecureCRT:/Prog/Sysinternals/cmd:/Prog/Sysinternals:/Prog/Common Files/DivX Shared:/Prog/QuickTime:/Prog/QuickTime/QTSystem:/Prog/NVIDIA Corporation/PhysX/Common:/Prog64/Razer Chroma SDK/bin:/Prog/Razer Chroma SDK/bin:/Windows/system32:/Windows:/Windows/System32/Wbem:/Prog64/Process Lasso:/usr/sbin:/Users/l.B/bin:/Users/l.B/bin/lib:/usr/local/bin:/bin:/etc/local/func_lib PERL_BADLANG (unset) SHELL=C:/Bin/Bash.exe ```

[p5pRT]

From @grinnz

This was an experimental feature added in 5.14​: https​://metacpan.org/pod/perl5140delta#Array-and-hash-container-functions-accept-references and then removed in 5.24​: https://metacpan.org/pod/perl5240delta#The-autoderef-feature-has-been-removed . One of the biggest problems with it is that there's no correct way to handle an object with both hash and array overloads defined.

[p5pRT]

The RT System itself - Status changed from 'new' to 'open'

[p5pRT]

From @grinnz

On Fri\, 13 Apr 2018 16​:43​:47 -0700\, grinnz@​gmail.com wrote​:

This was an experimental feature added in 5.14​: https://metacpan.org/pod/perl5140delta#Array-and-hash-container- functions-accept-references and then removed in 5.24​: https://metacpan.org/pod/perl5240delta#The-autoderef-feature-has-been- removed . One of the biggest problems with it is that there's no correct way to handle an object with both hash and array overloads defined.

Also\, adding an @​ to dereference it is a very simple workaround\, and will autovivify an undefined scalar used in this manner.

[p5pRT]

From @philiprbrenan

The initial experimental form was a good idea but too broad because of the interpretation of​:

for($a) {...}

But when it was removed the removal was perhaps too broad as well and perhaps should not have been removed from pop\, push\, shift\, unshift where it seems plausible that an extra @​ can be safely added. I thought that perhaps Linda's scheme would fail on​:

my $a; for(keys $a) {...}

but in this case the point is moot so the scheme survives. And in cases like​:

my %a = (0=>undef); push @​{$a{0}}\, 1; say STDERR dump(%a);

it would make code much more readable.

If Linda's scheme can be shown to be safe and unambiguous on all cases of pop\, push\, shift\, unshift then I would like to see this capability returned to Perl because it is more concise.

[p5pRT]

From @grinnz

On Fri\, 13 Apr 2018 17​:32​:21 -0700\, philiprbrenan@​gmail.com wrote​:

And in cases like​:

my %a = (0=>undef); push @​{$a{0}}\, 1; say STDERR dump(%a);

it would make code much more readable.

If Linda's scheme can be shown to be safe and unambiguous on all cases of pop\, push\, shift\, unshift then I would like to see this capability returned to Perl because it is more concise.

Just for reference\, this can also be written as push $a{0}->@​* on Perl 5.24+ (or slightly earlier with the experimental feature). That might be more palatable to some when it comes to long dereference chains.

[p5pRT]

From perl-diddler@tlinx.org

Dan Book via RT wrote​:

One of the biggest problems with it is that there's no correct way to handle an object with both hash and array overloads defined.

I disagree\, in so much as if there is ambiguity\, it is IMO\, inherently an error (incomplete specification) -- the computer cannot infer proper function by context.

[p5pRT]

From perl-diddler@tlinx.org

Philip R Brenan via RT wrote​:

The initial experimental form was a good idea but too broad because of the interpretation of​:

for($a) {...}

---

  ??   In an earlier writeup of this\, I deleted but should have left a key phrase​:

If there is no way to interpret the expression without an error being generated\, (a '@​' is expected)\, then allow it to be processed as a reference to an array.

However\, the above is legal perl code even if '$a' is "undef"\, as it should be processed as "for(undef){...}".

Since this is legal​:

  for ([1\,2\,3]\,[4\,5\,6]) {...}

(with loop run twice​: _=[1\,2\,3]\, then _=[4\,5\,6]).

then this must be treated the same way (and is)​:

  for ([1\,2\,3]) {...} (_=[1\,2\,3]). There can be no treatment of a ref as an array or hash in that context\, because the ref itself is a legal value.

  Are you saying there is some situation where somehow that for would be treated as​: for(@​$a){...}?

If so\, I don't see it. I.e. I don't see how there would be confusion. I.e. in a situation where it is value to treat the value as a scalar OR an array\, then the scalar would take precedence.

But when it was removed the removal was perhaps too broad as well and perhaps should not have been removed from pop\, push\, shift\, unshift where it seems plausible that an extra @​ can be safely added. I thought that perhaps Linda's scheme would fail on​:

my $a; for(keys $a) {...}

If '$a' is a ref to a hash​: my $a={one=>1\,two=>2}; for (keys $a) {...}\, then I don't see confusion\, as keys is an operator (not a bareword)\, that requires a hash\, right?

In that case\, treating it as "%$a" would seem the only valid interpretation.

but in this case the point is moot so the scheme survives. And in cases like​:

my %a = (0=>undef); push @​{$a{0}}\, 1; say STDERR dump(%a);

it would make code much more readable.

If Linda's scheme can be shown to be safe and unambiguous on all cases of pop\, push\, shift\, unshift then I would like to see this capability returned to Perl because it is more concise.

What other cases was it removed from?

I don't see a problem allowing it anywhere where​:

1) it would be an error to NOT dereference the ref\,   and 2) where there is no language ambiguity.

I'm sure this was thoroughly 'hashed' out at the time\, so maybe there are cases I'm not thinking of where I could be enlightened? That said\, though\, I feel there _can't_ be any such cases -- because of the above preconditions.

If there was ambiguity before\, and it was allowed\, how was it that it was not flagged as an error?

And if the code was already "legal"\, then dereferencing shouldn't be considered. (i.e. "for ($a) {...}" ).

So what am I missing? Thanks! Linda

[p5pRT]

From perl-diddler@tlinx.org

Philip R Brenan wrote​:

So far you do not appear to be missing anything. *for($a)* was ambiguous under the original implementation

---

  It should never be "ambiguous". According to the language\, it should always resolve to a single value. In shell\, using​:

prog='P "perl-%s​:"\, $]; my $aref = [ qw(one two three) ]; for ($aref) {P "%s"\, $_}'

then (in shell & various perl-bin dirs)​:

(probs building 5.6.2)

perl-5.8.9/usr/bin> PATH=$PWD​:$PATH perl -MP -CSA -e "$prog" perl-5.008009​: ["one"\, "two"\, "three"]

perl-5.10.1/usr/bin> PATH=$PWD​:$PATH perl -MP -CSA -e "$prog" perl-5.010001​: ["one"\, "two"\, "three"]

perl-5.12.5/usr/bin> PATH=$PWD​:$PATH perl -MP -CSA -e "$prog" perl-5.012005​: ["one"\, "two"\, "three"]

(had probs building 5.14)

perl -MP -CSA -e "$prog" perl-5.016003​: ["one"\, "two"\, "three"]

(5.18-5.22 not built/tested)

perl-5.24.0/usr/bin> PATH=$PWD​:$PATH perl -MP -CSA -e "$prog" perl-5.024000​: ["one"\, "two"\, "three"]

/perl-5.26.0/usr/bin> PATH=$PWD​:$PATH perl -MP -CSA -e "$prog" perl-5.026000​: ["one"\, "two"\, "three"]

(Note​: I have 'P' installed from cpan in the versions tested)

Except for the untested versions\, it seems like there is no ambiguity in "for ($aref) {...}"

Was the problem in one of the untested versions?

[p5pRT]

From perl-diddler@tlinx.org

Is perl in a nice version control system where a patch or code-push could be backed out if one wanted?

Or if a problem developed\, could it be bisected like the kernel dev tree?

[p5pRT]

From @doughera88

On Tue\, Apr 17\, 2018 at 05​:29​:58PM -0700\, L A Walsh wrote​:

Is perl in a nice version control system where a patch or code-push could be backed out if one wanted?

Yes\, see perldoc pod/perlhack.pod

Or if a problem developed\, could it be bisected like the kernel dev tree?

Yes\, see perldoc Porting/bisect-runner.pl

--   Andy Dougherty doughera@​lafayette.edu

[p5pRT]

From @xsawyerx

Hi Linda\,

Whether there are situations that are unambiguous for the compiler or not\, there are additional reasons to removing auto deref. This includes ambiguity for the developers as well - and even if that is arguable\, the confusion for the developer isn't. Providing an auto deref for some operations but not others is far worse than supporting it in full.

I think this ticket should be respectfully rejected.

On 04/14/2018 01​:31 AM\, Linda Walsh (via RT) wrote​:

# New Ticket Created by Linda Walsh # Please include the string​: [perl #133109] # in the subject line of all future correspondence about this issue. # \<URL​: https://rt-archive.perl.org/perl5/Ticket/Display.html?id=133109 >

This is a bug report for perl from perl-diddler@​tlinx.org\, generated with the help of perlbug 1.40 running under perl 5.26.1.

----------------------------------------------------------------- [Please describe your issue here]

If a scaler is null or an ARRAY ref\, why shouldn't perl be able to treat it as one in a context where a non-ARRAY ref would generate an error?

ex​: my $aref; push $aref\, 1;

I'm aware an alternate way to do this was added\, but perl\, like many languages often has multiple ways to do something. More so in perl in that a major feature in perl is autovivification.

This is a perfect example of perl autovivifying an array ref to use as an array needed by push. So why would it be "forbidden".

Even more curious is that perl is a permissive language. If you want to do something it will usually let you -- assuming that the user is in a position to better know what is wanted/needed in a specific circumstance. I remember a discussion of perl's OO design and how perl doesn't enforce class boundaries or the concept of 'private'\, by _default_. While extensions can can add such things\, nothing is forbidden. So why would something that used to work be made "forbidden"?

Can perl be allowed to do the right thing when perl already knows what it is?

Thanks\,

[Please do not change anything below this line] ----------------------------------------------------------------- --- Flags​: category=core severity=high --- Site configuration information for perl 5.26.1​:

Configured by ASSI at Tue Sep 26 18​:50​:45 CEST 2017.

Summary of my perl5 (revision 5 version 26 subversion 1) configuration​:

Platform​: osname=cygwin osvers=2.9.0(0.31853) archname=x86_64-cygwin-threads-multi uname='cygwin_nt-6.3 cygwin 2.9.0(0.31853) 2017-09-12 10​:18 x86_64 cygwin ' config_args='-des -Dprefix=/usr -Dmksymlinks -Darchname=x86_64-cygwin-threads -Dlibperl=cygperl5_26.dll -Dcc=gcc -Dld=g++ -Accflags=-ggdb -O2 -pipe -Wimplicit-function-declaration -fdebug-prefix-map=/mnt/share/maint/perl.x86_64/build=/usr/src/debug/perl-5.26.1-1 -fdebug-prefix-map=/mnt/share/maint/perl.x86_64/src/perl-5.26.1=/usr/src/debug/perl-5.26.1-1 -fwrapv' hint=recommended useposix=true d_sigaction=define useithreads=define usemultiplicity=define use64bitint=define use64bitall=define uselongdouble=undef usemymalloc=n default_inc_excludes_dot=define bincompat5005=undef Compiler​: cc='gcc' ccflags ='-DPERL_USE_SAFE_PUTENV -U__STRICT_ANSI__ -D_GNU_SOURCE -ggdb -O2 -pipe -Wimplicit-function-declaration -fdebug-prefix-map=/mnt/share/maint/perl.x86_64/build=/usr/src/debug/perl-5.26.1-1 -fdebug-prefix-map=/mnt/share/maint/perl.x86_64/src/perl-5.26.1=/usr/src/debug/perl-5.26.1-1 -fwrapv -fno-strict-aliasing -fstack-protector-strong -D_FORTIFY_SOURCE=2' optimize='-O3' cppflags='-DPERL_USE_SAFE_PUTENV -U__STRICT_ANSI__ -D_GNU_SOURCE -ggdb -O2 -pipe -Wimplicit-function-declaration -fdebug-prefix-map=/mnt/share/maint/perl.x86_64/build=/usr/src/debug/perl-5.26.1-1 -fdebug-prefix-map=/mnt/share/maint/perl.x86_64/src/perl-5.26.1=/usr/src/debug/perl-5.26.1-1 -fwrapv -fno-strict-aliasing -fstack-protector-strong' ccversion='' gccversion='6.4.0' gccosandvers='' intsize=4 longsize=8 ptrsize=8 doublesize=8 byteorder=12345678 doublekind=3 d_longlong=define longlongsize=8 d_longdbl=define longdblsize=16 longdblkind=3 ivtype='long' ivsize=8 nvtype='double' nvsize=8 Off_t='off_t' lseeksize=8 alignbytes=8 prototype=define Linker and Libraries​: ld='g++' ldflags =' -Wl\,--enable-auto-import -Wl\,--export-all-symbols -Wl\,--enable-auto-image-base -fstack-protector-strong' libpth=/usr/lib libs=-lpthread -lgdbm -ldb -ldl -lcrypt -lgdbm_compat perllibs=-lpthread -ldl -lcrypt libc=/usr/lib/libcygwin.a so=dll useshrplib=true libperl=cygperl5_26.dll gnulibc_version='' Dynamic Linking​: dlsrc=dl_dlopen.xs dlext=dll d_dlsymun=undef ccdlflags=' ' cccdlflags=' ' lddlflags=' --shared -Wl\,--enable-auto-import -Wl\,--export-all-symbols -Wl\,--enable-auto-image-base -fstack-protector-strong'

Locally applied patches​: Cygwin​: README Cygwin​: use auto-image-base instead of fixed DLL base address Cygwin​: modify hints Cygwin​: Configure correct libsearch Cygwin​: Configure correct libpth Cygwin​: Win32 correct UTF8 handling Perl​: File-Path-2.14 (fixes CVE2017-6512)

--- @​INC for perl 5.26.1​: /usr/local/lib/perl5/site_perl/5.26/x86_64-cygwin-threads /usr/local/share/perl5/site_perl/5.26 /usr/lib/perl5/vendor_perl/5.26/x86_64-cygwin-threads /usr/share/perl5/vendor_perl/5.26 /usr/lib/perl5/5.26/x86_64-cygwin-threads /usr/share/perl5/5.26

--- Environment for perl 5.26.1​: CYGWIN=system nodosfilewarning winsymlinks​:native export HOME=/Users/l.B LANG (unset) LANGUAGE (unset) LC_COLLATE=C LC_CTYPE=en_US.UTF-8 LC_MESSAGES=C LC_MONETARY=C LC_NUMERIC=C LC_TIME=C LD_LIBRARY_PATH (unset) LOGDIR (unset) PATH=.​:.​:/usr/bin​:/sbin​:/prog64/vim/current​:/Prog64/VanDyke Software/SecureCRT​:/Prog/Sysinternals/cmd​:/Prog/Sysinternals​:/Prog/Common Files/DivX Shared​:/Prog/QuickTime​:/Prog/QuickTime/QTSystem​:/Prog/NVIDIA Corporation/PhysX/Common​:/Prog64/Razer Chroma SDK/bin​:/Prog/Razer Chroma SDK/bin​:/Windows/system32​:/Windows​:/Windows/System32/Wbem​:/Prog64/Process Lasso​:/usr/sbin​:/Users/l.B/bin​:/Users/l.B/bin/lib​:/usr/local/bin​:/bin​:/etc/local/func_lib PERL_BADLANG (unset) SHELL=C​:/Bin/Bash.exe

[p5pRT]

From perl-diddler@tlinx.org

Sawyer X via RT wrote​:

Hi Linda\,

Whether there are situations that are unambiguous for the compiler or not\, there are additional reasons to removing auto deref. This includes ambiguity for the developers as well - and even if that is arguable\, the confusion for the developer isn't. Providing an auto deref for some operations but not others is far worse than supporting it in full.

I think this ticket should be respectfully rejected.

---

  Might I ask\, how it is confusing for a developer? Most languages have no typing indicator -- at least perl will flag an error if the variable being dereferenced is of the wrong type​:

perl -e 'my $a={}; push $a\,1;' Not an ARRAY reference at -e line 2.

  I'm not sure what you mean by providing for auto-deref in some operations but not others. It would be implemented in all operations where NOT auto-dereferencing would be an error.

  How would that be unclear for a perl developer who probably (right or wrong) knows what would be a syntax error and what would not be. It's an extension into an area that would previously have been *unused* -- because all attempts to use invalid syntax\, before\, would have been flagged with an error.

  But to make it easier\, it looks like perfunc holds the answer. It lists functions that work on ARRAYS and functions that work on HASHes.

  I may have missed something\, but the dereference feature would only work where the language would require an ARRAY or a HASH. If the language requires an ARRAY or a HASH\, the user would be able to supply that with a literal ARRAY or HASH\, OR a reference to an ARRAY or HASH.

  If you want to complain about it working in some places and not others -- it's based on the fact that the language only allows ARRAYs and HASHes in various locations.

  The manpage shows​:   Functions for real @​ARRAYs   "each"\, "keys"\, "pop"\, "push"\, "shift"\, "splice"\, "unshift"\,   "values"

  Functions for real %HASHes   "delete"\, "each"\, "exists"\, "keys"\, "values"

  As far as I can tell\, these should be the entirety of functions that have required a '@​' or '%' as part of the syntax. In those cases\, if a SCALAR holds a reference to an ARRAY or a HASH\, then the undereferenced scalar could be used without the need for adding extraneous characters because perl knows (and will flag an error) if a scalar does or doesn't contain a reference to the correct type.

  Similarly\, the open call can take a scalar with a value of 'undef' and can store a file reference in that scalar. Whether or not that will work is entirely dependent on the contents of that var -- if it contains any other type of reference or value\, it is specification says it should fail.

  Similarly\, in any location where an ARRAY or HASH is required\, that requirement can be satisfied only with a var of the correct type whether it be literal or a reference.

  The feature was enabled for 2 years\, and triggered warnings for the next 3 years before it was removed for no reason that I've been able to find out. Even the release notes claim that it was an experiment that "failed"\, but give no details about how or why it failed only that it was replaced with an optional syntax.

When one of the main complaints against perl (as being read by others or others learning it) has been that it looks like 'line noise'. Adding postfix sigils in addition to the more concise prefix sigils that were already there\, seems like the wrong direction to go. If anything\, perl should strongly think about going "sigil-optional" where there would be no ambiguity(no vars of the same name and would require being pre-defined or referenced)\, but that's somewhat of a different issue.

  I don't see this being some vague nebulous issue the way some seem to be painting it. In some discussions about this some expressed what they were told -- that some things were ambiguous. I really wanted to see an ambiguous case\, but so far\, no one has been able to present one (perhaps I didn't understand the extent to which the original concept had been applied).

  Perhaps Sawyer is thinking about something other than what this is?

[p5pRT]

From @grinnz

On Thu\, Apr 19\, 2018 at 6​:09 PM\, L A Walsh \perl\-diddler@&#8203;tlinx\.org wrote​: ...

The manpage shows​: Functions for real @​ARRAYs "each"\, "keys"\, "pop"\, "push"\, "shift"\, "splice"\, "unshift"\, "values"

  Functions for real %HASHes
      "delete"\, "each"\, "exists"\, "keys"\, "values"

each\, keys\, and values work on both arrays and hashes. As does exists and delete\, technically.

-Dan

[p5pRT]

From perl-diddler@tlinx.org

Dan Book via RT wrote​:

On Thu\, Apr 19\, 2018 at 6​:09 PM\, L A Walsh \perl\-diddler@&#8203;tlinx\.org wrote​: ...

The manpage shows​: Functions for real @​ARRAYs "each"\, "keys"\, "pop"\, "push"\, "shift"\, "splice"\, "unshift"\, "values"

  Functions for real %HASHes
      "delete"\, "each"\, "exists"\, "keys"\, "values"

each\, keys\, and values work on both arrays and hashes. As does exists and delete\, technically.

-Dan

Yes...for dereferencing to work in this case\, the scalar would have to hold either an ARRAY or HASH reference. It really makes no sense to try to use "each"\, "keys" nor "values" on a a scalar that has value 'undef'. It would make no sense and would (should) generate an error as it does now.

Only functions that require an ARRAY *XOR* HASH could take undef. Off hand\, I can't think of a function that would autovivify a HASH.

Does that clarify the intent of my proposal? It may be the case that my proposal is not the same as what was implemented before (I never used it)\, but only a subset that is deterministic.

[p5pRT]

From @iabyn

On Thu\, Apr 19\, 2018 at 06​:37​:26PM -0700\, L A Walsh wrote​:

Dan Book via RT wrote​:

On Thu\, Apr 19\, 2018 at 6​:09 PM\, L A Walsh \perl\-diddler@&#8203;tlinx\.org wrote​: ...

The manpage shows​: Functions for real @​ARRAYs "each"\, "keys"\, "pop"\, "push"\, "shift"\, "splice"\, "unshift"\, "values"

  Functions for real %HASHes
      "delete"\, "each"\, "exists"\, "keys"\, "values"

each\, keys\, and values work on both arrays and hashes. As does exists and delete\, technically.

-Dan

Yes...for dereferencing to work in this case\, the scalar would have to hold either an ARRAY or HASH reference. It really makes no sense to try to use "each"\, "keys" nor "values" on a a scalar that has value 'undef'. It would make no sense and would (should) generate an error as it does now.

But when you have a scalar reference which is overloaded\, perl can't know what overload method to call in those cases. For example​:

  use overload   '@​{}' => sub { print "array deref\n" ; return \@​real_a }\,   '%{}' => sub { print "hash deref\n" ; return \%real_h }\,   ;

  my $r = bless [];

  push $r\, 3; # prints "array derdef"\, modifies @​real_a   $r->{foo} = 'bar'; # prints "hash derdef"\, modifies %real_h   @​k = keys $r; # what overload method does this call?

What should perl do in the last case?

-- The optimist believes that he lives in the best of all possible worlds. As does the pessimist.

[p5pRT]

From @philiprbrenan

Presumably it would return () as $r is still a reference to an empty array despite the ingenious use of side effects implemented via operator overloading to manipulate other arrays and hashes and thereby give the impression that $r is both an array and a hash?

If push $r\, 3 were always interpreted identically to push @​$r\, 3 we would expect to get​:

  use Data​::Dump qw(dump);   my @​real_a;   my %real_h;   use overload   '@​{}' => sub { print "array deref\n" ; return \@​real_a }\,   '%{}' => sub { print "hash deref\n" ; return \%real_h }\,   ;

  my $r = bless [];

  push *@​$r*\, 3;   say STDERR "AAAA "\, dump($r); # AAAA bless([3]\, "main")   say STDERR "BBBB "\, dump(\@​real_a); # BBBB [3]

I.e. $r is not empty\, which illustrates that push $r and push @​$r would not always produce identical results under the proposed scheme in the presence of operator loading.

I urge that both of these problems can be overcome by Linda's statement of the the scheme​:

*Push gets its array either directly or via a reference. *In this example​:

push $r\, 3

would affect only $r\, not @​real_a\, because no array dereference would be implied or performed.

On Sat\, Apr 21\, 2018 at 4​:37 PM\, Dave Mitchell \davem@&#8203;iabyn\.com wrote​:

On Thu\, Apr 19\, 2018 at 06​:37​:26PM -0700\, L A Walsh wrote​:

Dan Book via RT wrote​:

On Thu\, Apr 19\, 2018 at 6​:09 PM\, L A Walsh \perl\-diddler@&#8203;tlinx\.org wrote​: ...

The manpage shows​: Functions for real @​ARRAYs "each"\, "keys"\, "pop"\, "push"\, "shift"\, "splice"\, "unshift"\, "values"

  Functions for real %HASHes
      "delete"\, "each"\, "exists"\, "keys"\, "values"

each\, keys\, and values work on both arrays and hashes. As does exists and delete\, technically.

-Dan

Yes...for dereferencing to work in this case\, the scalar would have to hold either an ARRAY or HASH reference. It really makes no sense to try to use "each"\, "keys" nor "values" on a a scalar that has value 'undef'. It would make no sense and would (should) generate an error as it does now.

But when you have a scalar reference which is overloaded\, perl can't know what overload method to call in those cases. For example​:

use overload
    '@​\{\}' => sub \{ print "array deref\\n" ; return \\@​real\_a \}\,
    '%\{\}' => sub \{ print "hash  deref\\n" ; return \\%real\_h \}\,
    ;

my $r = bless \[\];

push $r\, 3;        \# prints "array derdef"\, modifies @​real\_a
$r\->\{foo\} = 'bar'; \# prints "hash  derdef"\, modifies %real\_h
@​k = keys $r;      \# what overload method does this call?

What should perl do in the last case?

-- The optimist believes that he lives in the best of all possible worlds. As does the pessimist.

-- Thanks\,

Phil \<http​://www.appaapps.com/howToWriteAnApp.html>

Philip R Brenan \<http​://www.appaapps.com/howToWriteAnApp.html>

[p5pRT]

From @iabyn

On Sat\, Apr 21\, 2018 at 06​:17​:09PM +0100\, Philip R Brenan wrote​:

*Push gets its array either directly or via a reference. *In this example​:

push $r\, 3

would affect only $r\, not @​real_a\, because no array dereference would be implied or performed.

Then you've just broken overloading. For example\, suppose someone writes this innocuous method​:

  sub push_foo {   my ($self\, $array_ref) = @​_;   push $array_ref\, $self->foo;   }

this method works exactly as documented and expected\, until the day someone passes an overloaded scalar as an arg to it\, then finds that the overloading is being quietly ignored.

-- That he said that that that that is is is debatable\, is debatable.

[p5pRT]

From perl-diddler@tlinx.org

Dave Mitchell wrote​:

But when you have a scalar reference which is overloaded\, perl can't know what overload method to call in those cases. For example​:

use overload
    '@​\{\}' => sub \{ print "array deref\\n" ; return \\@​real\_a \}\,
    '%\{\}' => sub \{ print "hash  deref\\n" ; return \\%real\_h \}\,
    ;

my $r = bless \[\];

push $r\, 3;        \# prints "array derdef"\, modifies @​real\_a
$r\->\{foo\} = 'bar'; \# prints "hash  derdef"\, modifies %real\_h
@​k = keys $r;      \# what overload method does this call?

What should perl do in the last case?

---

Two answers. The first answer​: it never gets that far.

PERL5OPT="" perl /tmp/ovld.pl Not an unblessed ARRAY reference at /tmp/ovld.pl line 9.

2nd answer -- this was brought up by Dan Brook. I answered​:

-------- Original Message -------- Date​: Fri\, 13 Apr 2018 18​:47​:08 -0700 From​: L A Walsh

Dan Book wrote​:

One of the biggest problems with it is that there's no correct way to handle an object with both hash and array overloads defined.

---

I disagree\, in so much as if there is ambiguity\, it is IMO\, inherently an error (incomplete specification) -- the computer cannot infer proper function by context. ---- end Message -----

In any case where there is ambiguity\, the compiler must throw an error(like)​:

Ambiguous dereference @​ line 13​: Sigil required.

My suggestion was to auto-dereference references where perl knew the type of the reference. If it reference is ambiguous\, it can't know the type.

The 2nd part was to allow a scalar that holds 'undef' to be auto-vivified where only 1 solution would work. I.e.

my $ref push $ref\, 1;

would autovivify a temporary array pointed to by '$ref'\, as push requires an array.

Similarly​: my $ref; my $val=pop $ref;

would act identically to​:

my $val=pop @​$ref;

Any case where perl has an ambiguous choice\, must be flagged as an error. I can't see any other option being correct.

[p5pRT]

From @iabyn

On Sat\, Apr 21\, 2018 at 02​:10​:05PM -0700\, L A Walsh wrote​:

Two answers. The first answer​: it never gets that far.

PERL5OPT="" perl /tmp/ovld.pl Not an unblessed ARRAY reference at /tmp/ovld.pl line 9.

That's because specifically to get round the problem of the overloading ambiguity\, the perl autodedef ref implementation was changed so that it croaked on blessed refs. This made things even worse - you could never use the new syntax in any place where there was a possibility that the ref you get passed may have been blessed. This was a big motivation for eliminating the feature.

2nd answer -- this was brought up by Dan Brook. I answered​: In any case where there is ambiguity\, the compiler must throw an error(like)​:

Ambiguous dereference @​ line 13​: Sigil required.

My suggestion was to auto-dereference references where perl knew the type of the reference. If it reference is ambiguous\, it can't know the type.

Since perl almost never knows the type of a reference at compile-time\, how can this ever work? Can you give a more concrete example?

The 2nd part was to allow a scalar that holds 'undef' to be auto-vivified where only 1 solution would work. I.e.

my $ref push $ref\, 1;

would autovivify a temporary array pointed to by '$ref'\, as push requires an array.

Similarly​: my $ref; my $val=pop $ref;

would act identically to​:

my $val=pop @​$ref;

Any case where perl has an ambiguous choice\, must be flagged as an error. I can't see any other option being correct.

Do you mean as a compile-time error or a run-time error?

-- The Enterprise successfully ferries an alien VIP from one place to another without serious incident.   -- Things That Never Happen in "Star Trek" #7

[p5pRT]

From perl-diddler@tlinx.org

Dave Mitchell via RT wrote​:

On Sat\, Apr 21\, 2018 at 02​:10​:05PM -0700\, L A Walsh wrote​:

Two answers. The first answer​: it never gets that far.

PERL5OPT="" perl /tmp/ovld.pl Not an unblessed ARRAY reference at /tmp/ovld.pl line 9.

That's because specifically to get round the problem of the overloading ambiguity\, the perl autodedef ref implementation was changed so that it croaked on blessed refs. This made things even worse -

It would make the new perl incompatible with old code. One of the rules in the proposed change was that "push $_\,3" should be have identically to push @​$_\,3 at runtime. That change broke compatibility and doesn't really protect from the problem it was trying to fix since putting a '@​' in front overrides the restriction in the statement that died.

tperl P "hello from perl %s"\, $]; my $aref; bless $aref=[]\, "SomeClass"; push @​$aref\, 3; P "bye world\, aref = %s"\, $aref; ' hello from perl 5.016003 bye world\, aref = SomeClass[3] # \<-- result​: blessed array (CLASS[])   # w/3 pushed on

I'm guessing where this feature was really meant to "fix" things was in this statement​:   @​k = keys $r;

as I think I understand it\, only if the ref is in a class can it use overloading(?)\, and the bless-check stopped that in​:

Type of argument to keys on reference must be unblessed hashref or arrayref at /tmp/ovld.pl line 11.

Again\, I think checking if there was a bless on the ref was a bit of a hack\, since it "avoided" the problem rather than dealing with it.

In that scenario\, blessed or not\, the only way it could be resolved is if perl knew '$r' was one type or the other. Since it can't know\, the programmer needs to resolve that by adding a sigil (either statically\, or at runtime using an eval​:

my @​k; if ($wantref eq ARRAY) {   eval {@​k = keys @​$r}; } else {   eval {@​k = keys %$r}; }

or something similar. If you are looking for a compile time error message\, like the primitive syntax check of push/pop @​[$]ref forcing '@​'\, it can't really happen. Perl is a loosely typed language. It's usually some special case where it can type check at 'compile time'. But those special cases also become hard-coded limitations that can prohibit other options.

"

This made things even worse - you could never use the new syntax in any place where there was a possibility that the ref you get passed may have been blessed. This was a big motivation for eliminating the feature.

2nd answer -- this was brought up by Dan Brook. I answered​: In any case where there is ambiguity\, the compiler must throw an error(like)​:

Ambiguous dereference @​ line 13​: Sigil required.

My suggestion was to auto-dereference references where perl knew the type of the reference. If it reference is ambiguous\, it can't know the type.

Since perl almost never knows the type of a reference at compile-time\, how can this ever work? Can you give a more concrete example?

---

  It works for giving a runtime error just like​:

  my $r={};   push @​$r;

does now. It can't be detected until runtime if you want to use use runtime references. If forcing the addition of '@​' is important\, perhaps that could be toggled via a pragma\, but adding '@​' won't guarantee that it will work at runtime as in the example above. The compiler can only do static/syntax checks\, which may give warm fuzzies\, but may mislead some people if they think it prevents runtime errors.

Adding the "overload" module simply allow more options for runtime errors and in those cases\, the requirement for using explicit '@​' or '%' can not go away because it creates a runtime ambiguity that the runtime can't resolve and would have to flag as an error at runtime.

Do you mean as a compile-time error or a run-time error?

run-time -- just as the above example generates now.

No algorithm will work for all input. That doesn't mean you throw out division because you can't divide by 0.

[p5pRT]

From @xsawyerx

I'm currently in no position to have a long discussion on this so I'll just provide the short answer.

We tried autoderef. We removed it. We are not returning it.

On 04/20/2018 12​:09 AM\, L A Walsh wrote​:

Sawyer X via RT wrote​:

Hi Linda\,

Whether there are situations that are unambiguous for the compiler or not\, there are additional reasons to removing auto deref. This includes ambiguity for the developers as well - and even if that is arguable\, the confusion for the developer isn't. Providing an auto deref for some operations but not others is far worse than supporting it in full.   I think this ticket should be respectfully rejected.   ---    Might I ask\, how it is confusing for a developer?  Most languages have no typing indicator -- at least perl will flag an error if the variable being dereferenced is of the wrong type​:

perl -e 'my $a={}; push $a\,1;' Not an ARRAY reference at -e line 2.

   I'm not sure what you mean by providing for auto-deref in some operations but not others.  It would be implemented in all operations where NOT auto-dereferencing would be an error.

   How would that be unclear for a perl developer who probably (right or wrong) knows what would be a syntax error and what would not be.  It's an extension into an area that would previously have been *unused* -- because all attempts to use invalid syntax\, before\, would have been flagged with an error.

   But to make it easier\, it looks like perfunc holds the answer.  It lists functions that work on ARRAYS and functions that work on HASHes.

   I may have missed something\, but the dereference feature would only work where the language would require an ARRAY or a HASH.  If the language requires an ARRAY or a HASH\, the user would be able to supply that with a literal ARRAY or HASH\, OR a reference to an ARRAY or HASH.

   If you want to complain about it working in some places and not others -- it's based on the fact that the language only allows ARRAYs and HASHes in various locations.    The manpage shows​:       Functions for real @​ARRAYs           "each"\, "keys"\, "pop"\, "push"\, "shift"\, "splice"\, "unshift"\,           "values"

      Functions for real %HASHes           "delete"\, "each"\, "exists"\, "keys"\, "values"

   As far as I can tell\, these should be the entirety of functions that have required a '@​' or '%' as part of the syntax.  In those cases\, if a SCALAR holds a reference to an ARRAY or a HASH\, then the undereferenced scalar could be used without the need for adding extraneous characters because perl knows (and will flag an error) if a scalar does or doesn't contain a reference to the correct type.

   Similarly\, the open call can take a scalar with a value of 'undef' and can store a file reference in that scalar.  Whether or not that will work is entirely dependent on the contents of that var -- if it contains any other type of reference or value\, it is specification says it should fail.

   Similarly\, in any location where an ARRAY or HASH is required\, that requirement can be satisfied only with a var of the correct type whether it be literal or a reference.

   The feature was enabled for 2 years\, and triggered warnings for the next 3 years before it was removed for no reason that I've been able to find out.  Even the release notes claim that it was an experiment that "failed"\, but give no details about how or why it failed only that it was replaced with an optional syntax. When one of the main complaints against perl (as being read by others or others learning it) has been that it looks like 'line noise'.  Adding postfix sigils in addition to the more concise prefix sigils that were already there\, seems like the wrong direction to go.  If anything\, perl should strongly think about going "sigil-optional" where there would be no ambiguity(no vars of the same name and would require being pre-defined or referenced)\, but that's somewhat of a different issue.

   I don't see this being some vague nebulous issue the way some seem to be painting it.  In some discussions about this some expressed what they were told -- that some things were ambiguous.  I really wanted to see an ambiguous case\, but so far\, no one has been able to present one (perhaps I didn't understand the extent to which the original concept had been applied).    Perhaps Sawyer is thinking about something other than what this is?

[p5pRT]

From @iabyn

On Sat\, Apr 21\, 2018 at 07​:56​:48PM -0700\, L A Walsh wrote​:

No algorithm will work for all input. That doesn't mean you throw out division because you can't divide by 0.

The big difference between the two is that there's (in general) no practical alternative to using division\, while the auto-deref form can be avoided by the trivial syntactical addition of a '@​' or '%' to the source code\, plus possibly {} for precedence issues. Or use the recent posfix forms.

So there's a very modest gain for allowing push $r\, while there are many drawbacks\, requiring documentation for all the special cases\, and those special cases limiting the usefulness of the feature (like breaking overloading).

This feature\, and its removal\, has already been extensively discussed; it's problematical\, and its not going to be re-added.

-- Art is anything that has a label (especially if the label is "untitled 1")