Closed p5pRT closed 13 years ago
$ perl5.13.7 -le 'push 42\, 1\,2\,3; print "@42"' Type of arg 1 to push must be array (not constant item) at -e line 1\, near "3;" Execution of -e aborted due to compilation errors.
$ perl5.13.7 -le 'push ${\42}\, 1\,2\,3; print "@42"' 1 2 3
So\, with strict mode off\, symbolic deferencing is allowed unless it is detected at compile-time.
Let’s try something else:
$ perl5.13.7 -le 'use strict; push *f\, 1\,2\,3; print "@::f"' 1 2 3
$ perl5.13.7 -le 'use strict; use constant g=>*f; push g\, 1\,2\,3; print "@::f"' Type of arg 1 to push must be array (not constant item) at -e line 1\, near "3;" Execution of -e aborted due to compilation errors.
So push is sometimes stricter than strict.
The edge cases for keys are obviously even more weird:
$ perl5.13.7 -le '%foo = qw(a 1 b 1 c 1); $\,=" ";print keys foo =>' Hash %foo missing the % in argument 1 of keys() at -e line 1. c a b
$ perl5.13.7 -le '%foo = qw(a 1 b 1 c 1); $\,=" ";print keys "foo"' Type of argument to keys on reference must be hashref or arrayref at -e line 1.
I thought foo => and "foo" were equivalent.
$ perl5.13.7 -le '%foo = qw(a 1 b 1 c 1); $\,=" ";print keys *foo' Type of argument to keys on reference must be hashref or arrayref at -e line 1.
But *foo is a perfectly valid array/hash reference (even under strict). Why cannot it be resolved like a blessed scalar with @{} and %{} overloading?
More inconsistencies:
$ perl5.13.7 -le 'keys %$x = 7'
$ perl5.13.7 -le 'keys $x = 7'
Can't modify keys on reference in scalar assignment at -e line 1\, near "7;"
Execution of -e aborted due to compilation errors.
$ perl5.13.7 -le 'keys $x; print $x//"\
Also\, to bring this up again\, what if I have an object that is a blessed array (I do) the contents of which are part of the public API (they are) and the blessing exists for the sake of calling methods on it? What if I want to add %{} overloading to it as an alternate interface? In that case\, I am not changing the API (even by David Golden’s standards)\, as ref() returns the same thing. But that will break any code doing keys $object.
So\, could we please make this simpler? This set of rules for dealing with constants and overloading does not match anything elsewhere in Perl. So it is just another set of edge cases for people to learn.
Oh look\, there is a patch attached hereto.
(This also eliminates the undocumented ‘keys undef’ behaviour and makes keys $foo consistent with keys %$foo in terms of autovivification.)
Flags: category=core severity=high
Site configuration information for perl 5.13.7:
Configured by sprout at Thu Dec 9 14:53:58 PST 2010.
Summary of my perl5 (revision 5 version 13 subversion 7) configuration: Snapshot of: 9e9fdd5de1676d607ddf9b9f10b8df2659af3ded Platform: osname=darwin\, osvers=10.4.0\, archname=darwin-2level uname='darwin pint.local 10.4.0 darwin kernel version 10.4.0: fri apr 23 18:28:53 pdt 2010; root:xnu-1504.7.4~1release_i386 i386 ' config_args='-de -Dusedevel -DDEBUGGING' hint=recommended\, useposix=true\, d_sigaction=define useithreads=undef\, usemultiplicity=undef useperlio=define\, d_sfio=undef\, uselargefiles=define\, usesocks=undef use64bitint=undef\, use64bitall=undef\, uselongdouble=undef usemymalloc=n\, bincompat5005=undef Compiler: cc='cc'\, ccflags ='-fno-common -DPERL_DARWIN -no-cpp-precomp -DDEBUGGING -fno-strict-aliasing -pipe -fstack-protector -I/usr/local/include'\, optimize='-O3 -g'\, cppflags='-no-cpp-precomp -fno-common -DPERL_DARWIN -no-cpp-precomp -DDEBUGGING -fno-strict-aliasing -pipe -fstack-protector -I/usr/local/include' ccversion=''\, gccversion='4.2.1 (Apple Inc. build 5664)'\, gccosandvers='' intsize=4\, longsize=4\, ptrsize=4\, doublesize=8\, byteorder=1234 d_longlong=define\, longlongsize=8\, d_longdbl=define\, longdblsize=16 ivtype='long'\, ivsize=4\, nvtype='double'\, nvsize=8\, Off_t='off_t'\, lseeksize=8 alignbytes=8\, prototype=define Linker and Libraries: ld='env MACOSX_DEPLOYMENT_TARGET=10.3 cc'\, ldflags =' -fstack-protector -L/usr/local/lib' libpth=/usr/local/lib /usr/lib libs=-ldbm -ldl -lm -lutil -lc perllibs=-ldl -lm -lutil -lc libc=/usr/lib/libc.dylib\, so=dylib\, useshrplib=false\, libperl=libperl.a gnulibc_version='' Dynamic Linking: dlsrc=dl_dlopen.xs\, dlext=bundle\, d_dlsymun=undef\, ccdlflags=' ' cccdlflags=' '\, lddlflags=' -bundle -undefined dynamic_lookup -L/usr/local/lib -fstack-protector'
Locally applied patches:
@INC for perl 5.13.7: /usr/local/lib/perl5/site_perl/5.13.7/darwin-2level /usr/local/lib/perl5/site_perl/5.13.7 /usr/local/lib/perl5/5.13.7/darwin-2level /usr/local/lib/perl5/5.13.7 /usr/local/lib/perl5/site_perl .
Environment for perl 5.13.7: DYLD_LIBRARY_PATH (unset) HOME=/Users/sprout LANG=en_US.UTF-8 LANGUAGE (unset) LD_LIBRARY_PATH (unset) LOGDIR (unset) PATH=/usr/bin:/bin:/usr/sbin:/sbin:/usr/local/bin:/usr/X11/bin:/usr/local/bin PERL_BADLANG (unset) SHELL=/bin/bash
[from the tail end of the report]
Oh look\, there is a patch attached hereto.
That's a very disingenuous (not to mention sarcastic) way to submit a "revert a feature" patch.
I understand that you don't like aspects of this feature. However\, the opportunity to comment on the addition of this feature was a while ago and the debates were quite long and there was plenty of opportunity for input before the decision was taken to add it.
You cite a handful of edge cases. I'd be happy to collaborate with you on fixing the edge cases\, but I don't find your current approach to be constructive.
Regards\, David
The RT System itself - Status changed from 'new' to 'open'
On Dec 12\, 2010\, at 2:02 PM\, David Golden via RT wrote:
[from the tail end of the report]
Oh look\, there is a patch attached hereto.
That's a very disingenuous (not to mention sarcastic) way to submit a "revert a feature" patch.
I was afraid it might be taken that way. I apologise sincerely if I have hurt your feelings.
I understand that you don't like aspects of this feature. However\, the opportunity to comment on the addition of this feature was a while ago and the debates were quite long and there was plenty of opportunity for input before the decision was taken to add it.
There were only four days between the submission of your patch and its application.
I do not want to get dragged into an argument. So I’ve asked Jesse to decide which version we will have.
On 12 December 2010 21:21\, Father Chrysostomos \perlbug\-followup@​perl\.org wrote:
# New Ticket Created by Father Chrysostomos # Please include the string: [perl #80626] # in the subject line of all future correspondence about this issue. # \<URL: http://rt.perl.org/rt3/Ticket/Display.html?id=80626 >
$ perl5.13.7 -le 'push 42\, 1\,2\,3; print "@42"' Type of arg 1 to push must be array (not constant item) at -e line 1\, near "3;" Execution of -e aborted due to compilation errors.
$ perl5.13.7 -le 'push ${\42}\, 1\,2\,3; print "@42"' 1 2 3
So\, with strict mode off\, symbolic deferencing is allowed unless it is detected at compile-time.
Let’s try something else:
$ perl5.13.7 -le 'use strict; push *f\, 1\,2\,3; print "@::f"' 1 2 3
$ perl5.13.7 -le 'use strict; use constant g=>*f; push g\, 1\,2\,3; print "@::f"' Type of arg 1 to push must be array (not constant item) at -e line 1\, near "3;" Execution of -e aborted due to compilation errors.
So push is sometimes stricter than strict.
The edge cases for keys are obviously even more weird:
$ perl5.13.7 -le '%foo = qw(a 1 b 1 c 1); $\,=" ";print keys foo =>' Hash %foo missing the % in argument 1 of keys() at -e line 1. c a b
$ perl5.13.7 -le '%foo = qw(a 1 b 1 c 1); $\,=" ";print keys "foo"' Type of argument to keys on reference must be hashref or arrayref at -e line 1.
I thought foo => and "foo" were equivalent.
$ perl5.13.7 -le '%foo = qw(a 1 b 1 c 1); $\,=" ";print keys *foo' Type of argument to keys on reference must be hashref or arrayref at -e line 1.
But *foo is a perfectly valid array/hash reference (even under strict). Why cannot it be resolved like a blessed scalar with @{} and %{} overloading?
More inconsistencies:
$ perl5.13.7 -le 'keys %$x = 7' $ perl5.13.7 -le 'keys $x = 7' Can't modify keys on reference in scalar assignment at -e line 1\, near "7;" Execution of -e aborted due to compilation errors. $ perl5.13.7 -le 'keys $x; print $x//"\
"' \ $ perl5.13.7 -le 'push $x; print $x//"\ "' ARRAY(0x803a00) Also\, to bring this up again\, what if I have an object that is a blessed array (I do) the contents of which are part of the public API (they are) and the blessing exists for the sake of calling methods on it? What if I want to add %{} overloading to it as an alternate interface? In that case\, I am not changing the API (even by David Golden’s standards)\, as ref() returns the same thing. But that will break any code doing keys $object.
So\, could we please make this simpler? This set of rules for dealing with constants and overloading does not match anything elsewhere in Perl. So it is just another set of edge cases for people to learn.
On one hand I think a lot of your comments are important edge cases that we need to address. In particular keys $hash= 7; jumped out at me.
On the other hand\, I dont buy the simple rules argument. I think Davids rules DWIM pretty reasonably. People that expect a specific interpretation can specify which behaviour they expect.
I feel that it would be pity if we lost this stuff because of an edge case where only a minimal number of "deeper" perl hackers would go anyway. I cannot remember the last time I saw robust production code which used reference overloading.
Is there some "third way" where you can have your desired and admittedly more consistent and robust semantics without such a drastic change to his patch? A pragma maybe?
I know you don't want to argue\, and wanted Jesse to make a ruling\, so lets say he says "The overload DWIM sticks\, but the edge cases have to be resolved" what would suggest is the right way to get them resolved?
How would you feel about writing a patch to add tests for all the crazy edge cases you have found so that David can use them to eliminate as many edge cases as possible.
Which would leave the only open question "what to do about overloading" and can be decided by Jesse when he has more time.
cheers\, Yves
-- perl -Mre=debug -e "/just|another|perl|hacker/"
demerphq wrote:
Is there some "third way" where you can have your desired and admittedly more consistent and robust semantics without such a drastic change to his patch? A pragma maybe?
The least-crazy way to keep keys($ref) is for it to always mean keys(%$ref). It's only the rarely-used array-viewed-like-a-hash operations that cause hassle here.
-zefram
On Mon\, Dec 13\, 2010 at 6:11 PM\, Zefram \zefram@​fysh\.org wrote:
demerphq wrote:
Is there some "third way" where you can have your desired and admittedly more consistent and robust semantics without such a drastic change to his patch? A pragma maybe?
The least-crazy way to keep keys($ref) is for it to always mean keys(%$ref). It's only the rarely-used array-viewed-like-a-hash operations that cause hassle here.
In the case where $ref is an unblessed arrayref\, I think it should still work like @$ref. However\, in an ambiguous case where $ref is blessed and both @{} and %{} are possible (due to underlying reftype or due to overloading)\, always defaulting to %{} (with a warning) is the other very reasonable approach. (I see that or the current behavior as equally arbitrary\, so defer to whatever seems to make the most sense to people.)
-- David
David Golden wrote:
In the case where $ref is an unblessed arrayref\, I think it should still work like @$ref.
That's where the craziness starts. In keys(%$ref) I might have been relying on it dying if my caller passed me something other than a hash ref. If I can't rely on keys($ref) having the same behaviour then this is something extra that I have to bear in mind and check before using keys($ref). Which means I can't let myself get into the habit of casually using keys($ref) in place of keys(%$ref). Which rather destroys the point of the shorthand.
-zefram
On Tue\, Dec 14\, 2010 at 08:44:08AM +0000\, Zefram wrote:
David Golden wrote:
In the case where $ref is an unblessed arrayref\, I think it should still work like @$ref.
That's where the craziness starts. In keys(%$ref) I might have been relying on it dying if my caller passed me something other than a hash ref. If I can't rely on keys($ref) having the same behaviour then this is something extra that I have to bear in mind and check before using keys($ref). Which means I can't let myself get into the habit of casually using keys($ref) in place of keys(%$ref). Which rather destroys the point of the shorthand.
So\, you keep using 'keys(%$ref)'\, and the proposed behaviour of 'keys($ref)' won't get in your way.
I find "but the old way dies\, and code may rely on that" a pretty stiffling argument. It basically means\, nothing can ever change - one cannot introduce a construct that's invalid in older perls\, cause there may be some code out there that relies on it currently dying.
The savings of a single keystroke isn't something that I find really exciting\, but I do appreciate the proposal\, and I can see that there are people who prefer to have less sigils. And it's a matter of more DWIM\, which I always welcome.
Abigail
On Mon\, Dec 13\, 2010 at 5:43 PM\, demerphq \demerphq@​gmail\.com wrote:
Which would leave the only open question "what to do about overloading" and can be decided by Jesse when he has more time.
[This is a moderately pedantic post -- those with limited time (Hi\, Jesse!) might want to jump to "CONCLUSIONS" at the bottom]
I want to speak to the overloaded objects ambiguities concern and lay out a spectrum of possible behaviors. I'll use "keys $ref" for these examples\, though it applies equally to values and each.
Here are several potential approaches:
(0) Do not allow "keys $ref" at all. It would be a syntax error just as in 5.12.
(1) Allow "keys $ref"\, but always interpret it as "keys %{$ref}".
(2) Allow "keys $ref"\, but only for unblessed references. Dereference based on reftype. Die if $ref is an object.
(3) Allow "keys $ref"\, but only for unblessed references or objects without %{} and @{} overloading. Dereference based on reftype. Die if $ref is an object with dereference overloading.
(4) Allow "keys $ref"\, but only for unblessed references\, objects without overloading\, objects where the reftype matches dereference overloading or objects where the reftype is neither HASH nor ARRAY and where there is only one of %{} and @{}. Die if $ref is a blessed hash with @{} overloading; a blessed array with %{} overloading; a blessed anything with both @{} and %{}; or a blessed non-container without one of @{} or %{} overloading. Dereference based on overloading if present or reftype otherwise.
All of the cases 1 - 4 above have no ambiguity in interpretation. The reference that gets dereferenced only appears to be of one reftype whether naturally or synthetically via overloading. The next case and its variations introduce the need to resolve ambiguity:
(5) Allow "keys $ref" for any reference or object\, regardless of overloading. If there is no overloading\, or if overloading matches the reftype\, or if only one of %{} and @{} overloading exists for a non-container reftype object\, then dereference accordingly (unambiguous). If there is ambiguity\, warn about the ambiguity and see (5a) through (5e) for resolution:
(5a) For ambiguity\, dereference with %{} always
(5b) For ambiguity\, dereference according to the reftype for ARRAY and HASH; for non-container reftypes\, dereference based on overloading\, but favor @{} if there are both
(5c) For ambiguity\, dereference according to the reftype for ARRAY and HASH; for non-container reftypes\, dereference based on overloading\, but favor %{} if there are both
(5d) For ambiguity\, dereference based on overloading if there is only one of @{} or %{} and as @{} if there are both overloadings
(5e) For ambiguity\, dereference based on overloading if there is only one of @{} or %{} and as %{} if there are both overloadings
For reference\, the patch that is now in blead equates to (5e) above. I think all of (5a) to (5e) are relatively arbitrary choices\, which is why I think they should issue warnings. I tend to favor (5e)\, (5a) and (5c) -- in that order -- as the most sensible options among them\, which are all variations of "favor hashes" to various degrees.
To step back a bit\, I see an interesting sharp line between (2) and (3) above. Option (2) treats objects as opaque whereas options (3) and beyond break encapsulation. That could be an interesting place to draw the line because it's not unreasonable to require people breaking encapsulation to be explicit about it.
Between (3) and (4) is a step that maximizes the feature whenever there is only one way to dereference. However\, I acknowledge the risk of the sort of surprise API breakage that concerns Father C. Adding a feature (some sort of overloaded dereferencing) to a class could cause previously working code to just die.
Moving from (4) to (5) is just a less harsh approach than (4)\, in that Perl attempts to do something reasonable (for some definition of "reasonable") in ambiguous cases and issues a warning about it. Assuming that the warnings can be made fatal at the users option\, I think it's more perlish to warn instead of die and let the stricter behavior be an option for the user to enable if desired. Thus\, I think (5) is clearly preferable to (4).
CONCLUSION
There is a lot of code that just manipulates simple data structures entirely within the boundaries of a programmer's control where no ambiguity is possible. I would be disappointed to see "keys $ref" go away for those common cases because of the possibility of surprise in what I expect are much rarer cases where overloading is being used to make objects have polymorphic behavior.
Given the cases I describe above\, I see four of them as preferred options (which I'll restate for those who jumped to the conclusion):
(2) Defend encapsulation: have "keys $ref" work only for unblessed references and die otherwise. No ambiguity is possible.
(5a) Always guess hash: Have "keys $ref" work for blessed or unblessed references; in ambiguous cases\, warn and dereference with %{} always.
(5c) Guess by reftype: Have "keys $ref" work for blessed or unblessed references; in ambiguous cases\, warn and dereference according to the reftype for ARRAY and HASH; for non-container reftypes\, dereference based on overloading\, but favor %{} if there are both
(5e) Guess by overload: Have "keys $ref" work for blessed or unblessed references; in ambiguous cases\, warn and dereference based on overloading if there is only one of @{} or %{} and as %{} if there are both overloadings
I chose (5e) for the original patch because it seemed the most liberal yet consistent (i.e. not based on whether something had a container reftype or not)\, but I would be happy with any of these four behaviors as the final result.
-- David
* David Golden \xdaveg@​gmail\.com [2010-12-15T09:24:22]
[ ... ]
Thanks for this post.
(2) Defend encapsulation: have "keys $ref" work only for unblessed references and die otherwise. No ambiguity is possible.
I think this is the best way forward\, of those in your conclusion. It didn't work before\, it doesn't work now\, but there's still benefit for simple containers. In the future\, overloading could provide disambiguation: you may return an object if it marked with the overloading that says "acts like an array for push/pop/shift/unshift\," for example. Or\, alternately\, that says\, "in ^-context\, act like this kind of reference always."
The problem case that FC mentioned still exists: You used to return an arrayref\, and people relied on calling (say) keys on it\, but now you want to return a blessed arrayref. With protected encapsulation\, without the overloading existing\, this will fail.
Although I am excited by "push $aref\, @values" I think I am back where I spent much of my time watching this feature: in favor of (0)\, you can't push onto a reference. The alternate proposal then was better syntax for dereference\, which has the added benefit of working in many\, many other places\, and of being entirely unambiguous.
Is has the significant failing of not yet having been implemented.
I think I still find these much better in the long run:
push Array $foo->bar\, @items; # or push $foo->bar->@\, @items;
...and that if it means we spend another year having to wrap things in @{...}\, then that's a price I'm willing to pay.
-- rjbs
On Wed\, Dec 15\, 2010 at 9:24 AM\, David Golden \xdaveg@​gmail\.com wrote:
I want to speak to the overloaded objects ambiguities concern and lay out a spectrum of possible behaviors. I'll use "keys $ref" for these examples\, though it applies equally to values and each.
Here are several potential approaches:
(0) Do not allow "keys $ref" at all. It would be a syntax error just as in 5.12.
(1) Allow "keys $ref"\, but always interpret it as "keys %{$ref}".
Hi\,
I'm against adding more polymorphism based on the current representation of the value. It goes against Perl's design\, and it's has been proven a source of problems time and time again. For example\, The Unicode Bug\, pseudo-hashes\, globs vs scalars with globs\, bit operators and even @INC.
The effects are even predictable. How often to you think "keys $ref" will be used with the intention of preventing overloading? Yet every single option except (0) and (1) does just that. It was implied that programmers would know to avoid the unqualified version for code that could receive overloaded objects and would avoid it in those circumstances\, but that's simply not true.
- Eric Brine
On Wed\, 15 Dec 2010\, David Golden wrote:
Here are several potential approaches:
(0) Do not allow "keys $ref" at all. It would be a syntax error just as in 5.12.
(1) Allow "keys $ref"\, but always interpret it as "keys %{$ref}".
(2) Allow "keys $ref"\, but only for unblessed references. Dereference based on reftype. Die if $ref is an object.
I'm in favor of (1). It is a really simple rule: "keys/values/each work on hashes\, arrays\, and hash references"\, and it is obvious how it interacts with overloading.
So it doesn't work with array references\, but then\, it never used to do so before\, and in some ways having keys/values/each work on arrays is of questionable value to begin with (IMHO).
Cheers\, -Jan
On Wed\, Dec 15\, 2010 at 09:24:22AM -0500\, David Golden wrote:
On Mon\, Dec 13\, 2010 at 5:43 PM\, demerphq \demerphq@​gmail\.com wrote:
Which would leave the only open question "what to do about overloading" and can be decided by Jesse when he has more time.
[This is a moderately pedantic post -- those with limited time (Hi\, Jesse!) might want to jump to "CONCLUSIONS" at the bottom]
I want to speak to the overloaded objects ambiguities concern and lay out a spectrum of possible behaviors. I'll use "keys $ref" for these examples\, though it applies equally to values and each.
Here are several potential approaches:
(0) Do not allow "keys $ref" at all. It would be a syntax error just as in 5.12.
(1) Allow "keys $ref"\, but always interpret it as "keys %{$ref}".
This would be my option. Because a) I think this will be what most programmers want it do\, and\, more importantly\, b) you can describe that behaviour in a single line. 4) and 5) add\, IMO\, too much complexity (not in the sense of implementation\, but of the language) for too little gain. And that's probably true for 2) and 3) as well.
Abigail
On Wed\, Dec 15\, 2010 at 09:24:22AM -0500\, David Golden wrote: > Here are several potential approaches:
(0) Do not allow "keys $ref" at all. It would be a syntax error just as in 5.12.
(1) Allow "keys $ref"\, but always interpret it as "keys %{$ref}".
This one.
But my bias is against keys etc on arrays anyway.
Tony
On Mon Dec 13 14:43:36 2010\, demerphq wrote:
How would you feel about writing a patch to add tests for all the crazy edge cases you have found so that David can use them to eliminate as many edge cases as possible.
I am willing to do that\, but I’ve been very busy lately and it looks as though I will not have a chance for a week and a half. The edge cases for keys will be determined by whether the current overloading behaviour will stay\, and I still have to think through all the edge cases if it does.
On Mon Dec 13 14:43:36 2010\, demerphq wrote:
How would you feel about writing a patch to add tests for all the crazy edge cases you have found so that David can use them to eliminate as many edge cases as possible.
I am willing to do that\, but I’ve been very busy lately and it looks as though I will not have a chance for a week and a half. The edge cases for keys will be determined by whether the current overloading behaviour will stay\, and I still have to think through all the edge cases if it does.
On Wed\, Dec 15\, 2010 at 09:24:22AM -0500\, David Golden wrote:
I want to speak to the overloaded objects ambiguities concern and lay out a spectrum of possible behaviors. I'll use "keys $ref" for these examples\, though it applies equally to values and each. [big snip]
This thread has been silent for 3 months\, and we're now one day away from the notional 5.14.0 code freeze\, so I'm wondering what we should do.
There seem to be two issues:
a) whether to always treat $r as a *hash* ref in 'keys $r' etc (as opposed to doing more complex stuff to determine whether to execute it as 'keys @$r' or 'keys %$r'). There seemed to be a rough consensus on this approach.
b) FC identified a number of edge cases\, where the thing after keys etc wasn't a simple scalar variable\, but a glob\, or a '=>'-quoted string etc. There doesn't seem to be much open for discussion there\, its more just a case of fixing them.
Now\, can you (DG) and/or us (p5p) fix (a) and/or (b) before the code-freeze\, and if not\, should this feature be removed for now\, and re-addressed post-5.14??
I think I'd reluctantly vote for removal.
-- "But Sidley Park is already a picture\, and a most amiable picture too. The slopes are green and gentle. The trees are companionably grouped at intervals that show them to advantage. The rill is a serpentine ribbon unwound from the lake peaceably contained by meadows on which the right amount of sheep are tastefully arranged." -- Lady Croom\, "Arcadia"
On Sat\, Mar 19\, 2011 at 6:57 PM\, Dave Mitchell \davem@​iabyn\.com wrote:
Now\, can you (DG) and/or us (p5p) fix (a) and/or (b) before the code-freeze\, and if not\, should this feature be removed for now\, and re-addressed post-5.14??
I think I'd reluctantly vote for removal.
We're past the freeze for "user visible changes"\, which (IMO) would include removal of a feature.
The DWIM behavior decision is ultimately Jesse's and a number of options were laid out and an opinions presented (on and off list). If Jesse isn't ready to make a decision\, maybe we document the feature as "experimental" for 5.14\, which gives him wiggle room to decide later.
The edge case fixes were blocking on TODO tests for them. Father C was hoping to get to them when time permitted. Demerphq's assessment seemed to be that they might only impact "deeper" perl hackers in any case. I think those should just be fixed in 5.15 and (optionally) backported to 5.14
-- David
On Tue\, Dec 14\, 2010 at 12:11 AM\, Zefram \zefram@​fysh\.org wrote:
The least-crazy way to keep keys($ref) is for it to always mean keys(%$ref). It's only the rarely-used array-viewed-like-a-hash operations that cause hassle here.
-zefram
Agreed. Please let this be the behaviour. No implicit dereferencing arrays with keys or values.
On Sun\, Mar 20\, 2011 at 1:44 AM\, David Golden \xdaveg@​gmail\.com wrote:
We're past the freeze for "user visible changes"\, which (IMO) would include removal of a feature.
The DWIM behavior decision is ultimately Jesse's and a number of options were laid out and an opinions presented (on and off list). If Jesse isn't ready to make a decision\, maybe we document the feature as "experimental" for 5.14\, which gives him wiggle room to decide later.
I'd be okay with that.
Ambrus
On Sat 19.Mar'11 at 20:44:37 -0400\, David Golden wrote:
On Sat\, Mar 19\, 2011 at 6:57 PM\, Dave Mitchell \davem@​iabyn\.com wrote:
Now\, can you (DG) and/or us (p5p) fix (a) and/or (b) before the code-freeze\, and if not\, should this feature be removed for now\, and re-addressed post-5.14??
I think I'd reluctantly vote for removal.
We're past the freeze for "user visible changes"\, which (IMO) would include removal of a feature.
The DWIM behavior decision is ultimately Jesse's and a number of options were laid out and an opinions presented (on and off list). If Jesse isn't ready to make a decision\, maybe we document the feature as "experimental" for 5.14\, which gives him wiggle room to decide later.
I suppose that marking it as experimental does give us the wiggle room to deal with it after 5.14\, but I worry that it's a "behavior" rather than a feature. I'd sort of been waiting to see the testing of edge cases get sorted out. How plausible would it be to hide this feature behind a "use feature" directive for 5.14?
-j
The edge case fixes were blocking on TODO tests for them. Father C was hoping to get to them when time permitted. Demerphq's assessment seemed to be that they might only impact "deeper" perl hackers in any case. I think those should just be fixed in 5.15 and (optionally) backported to 5.14
-- David
On Mar 20\, 2011\, at 4:30 PM\, Jesse via RT wrote:
On Sat 19.Mar'11 at 20:44:37 -0400\, David Golden wrote:
On Sat\, Mar 19\, 2011 at 6:57 PM\, Dave Mitchell \davem@​iabyn\.com wrote:
Now\, can you (DG) and/or us (p5p) fix (a) and/or (b) before the code-freeze\, and if not\, should this feature be removed for now\, and re-addressed post-5.14??
I think I'd reluctantly vote for removal.
We're past the freeze for "user visible changes"\, which (IMO) would include removal of a feature.
The DWIM behavior decision is ultimately Jesse's and a number of options were laid out and an opinions presented (on and off list). If Jesse isn't ready to make a decision\, maybe we document the feature as "experimental" for 5.14\, which gives him wiggle room to decide later.
I suppose that marking it as experimental does give us the wiggle room to deal with it after 5.14\, but I worry that it's a "behavior" rather than a feature.
I'd sort of been waiting to see the testing of edge cases get sorted out.
I’m afraid I had forgotten about the edge cases. Now that I think about it though\, we have a chicken-and-egg problem. There is no problem with push\, splice\, etc.\, as the edge cases are all currently errors and can be resolved later without backward compatibility concerns. The edges cases for keys\, each and values\, however\, entirely depend upon your decision with regard to array refs and objects.
How plausible would it be to hide this feature behind a "use feature" directive for 5.14?
That would be possible\, but wouldn’t it be strange to put it there? What would the same ‘feature’ feature do in 5.16?
I think marking that aspect of those functions as experimental in the docs (and removing the ‘or ARRAYREF’ from the =item directives in perlfunc) would be acceptable.
-j
The edge case fixes were blocking on TODO tests for them. Father C was hoping to get to them when time permitted. Demerphq's assessment seemed to be that they might only impact "deeper" perl hackers in any case. I think those should just be fixed in 5.15 and (optionally) backported to 5.14
-- David
On Mon Mar 21 20:40:19 2011\, sprout wrote:
On Mar 20\, 2011\, at 4:30 PM\, Jesse via RT wrote:
I'd sort of been waiting to see the testing of edge cases get sorted out.
I’m afraid I had forgotten about the edge cases. Now that I think about it though\, we have a chicken-and-egg problem. There is no problem with push\, splice\, etc.\, as the edge cases are all currently errors and can be resolved later without backward compatibility concerns. The edges cases for keys\, each and values\, however\, entirely depend upon your decision with regard to array refs and objects.
Attached is a first go at a test script. Edit the JC constant to choose between the different behaviours tested for.
BTW\, I noticed that ck_push is duplicating some logic from ck_fun. That was why in my first patch to this ticket I deleted ck_push and added some code to the array section of ck_fun.
#!/usr/bin/perl
# For most of these tests\, I���m actually trying to make the behaviour # comparable to what comes before ->[...] and ->{...}. Note that # "foo"->{...} is permitted even under strict refs (strict vars\, not refs\, # catches it)\, as the name is known at compile time. Hence "INC"->{...} and # 42->[...] are always permitted\, as they are globals.
# I���m not entirely sure about the edge cases for keys for 5a-5e.
# This script as-is should not be incorporated into the perl core\, but # rather tests from it should be incorporated into other test scripts\, # whichever are appropriate.
# What the tests test for are based on Jesse���s Choice: sub JC (){ '1' } # (please) # which is one of David Golden���s options: # (0) Do not allow keys "$ref" at all. # (1) keys $ref always means keys %$ref # (2) keys $ref only on unblessed refs # (3) keys $ref only on non-deref-overloaded refs # (4) keys $ref for hashes and arrays\, but not arrays with %{}\, # hashes with @{}\, or other objects with both @{} and %{} # or neither # (5) keys $ref on any reference or object\, resolving ambiguities # like this: # (5a) use %{} # (5b) based on underlying type for real arrays and hashes\, # based on overloading otherwise\, favouring @{} # (5c) based on underlying type for real arrays and hashes\, # based on overloading otherwise\, favouring %{} # (5d) dereference based on overloading\, when present\, favouring # @{} when both overloading types are present # (5e) dereference based on overloading\, when present\, favouring # %{} when both overloading types are present
BEGIN { die "Why are you even running this?"if!JC } BEGIN { die "Specify the suboption\, please" if JC eq 5 } BEGIN { die "Invalid choice" if JC !~ /^(?:[1-4]|5[a-e])\z/}
use Test::More; plan tests => 19; use strict;
# push with various non-ref scalars eval 'push 42\, 1\,2\,3'; is "@42"\, "1 2 3"\, 'push 42'; ok !eval 'push ${\42}\, 1\,2\,3; 1' && $@ =~ /strict refs/\, 'push $expr is subject to strict refs'; ok !eval 'push "foo"\, 1\,2\,3; 1' && $@ =~ /explicit package name/\, 'push "string" is subject to strict vars';
# push with globs eval 'push *bar\, 1\,2\,3'; is "@::bar"\, "1 2 3"\, 'push *bar'; use constant bazglob => *baz; eval 'push bazglob\, 1\,2\,3'; is "@::baz"\, "1 2 3"\, 'push glob_constant';
# keys with various non-ref scalars @42 = (1\,2\,3); %42 = qw(a 1 b 1 c 1); my @keys = eval 'sort keys 42'; is "@keys"\, JC =~ /[234]/ ? "" : JC =~ /1|5[ace]/ ? "a b c" : "1 2 3"\, 'keys 42 when both @42 and %42 exist'; ok JC =~ /[234]/ ? $@ : !$@\, '(no) error after keys 42'; our %hash = %42; our @array = @42; @keys = eval 'sort keys "hash"'; is "@keys"\, JC =~ /[234]/ ? "" : "a b c"\, 'keys "hash" when only %hash exists'; ok JC =~ /[234]/ ? $@ : !$@\, '(no) error after keys "hash"'; @keys = eval 'sort keys "array"'; is "@keys"\, JC =~ /[1-4]|5a/ ? "" : "1 2 3"\, 'keys "array" when only @array exists'; ok JC =~ /[234]/ ? $@ : !$@\, '(no) error after keys "array"';
# keys with globs @keys = eval 'sort keys *42'; is "@keys"\, JC =~ /[234]/ ? "" : JC =~ /1|5[ace]/ ? "a b c" : "1 2 3"\, 'keys *42 when both @42 and %42 exist'; ok JC =~ /[234]/ ? $@ : !$@\, '(no) error after keys *42'; @keys = eval 'sort keys *hash'; is "@keys"\, JC =~ /[234]/ ? "" : "a b c"\, 'keys *hash when only %hash exists'; ok JC =~ /[234]/ ? $@ : !$@\, '(no) error after keys *hash'; @keys = eval 'sort keys *array'; is "@keys"\, JC =~ /[1-4]|5a/ ? "" : "1 2 3"\, 'keys *array when only @array exists'; ok JC =~ /[234]/ ? $@ : !$@\, '(no) error after keys *array';
# lvalue keys my $aref = []; my $href = {}; ok !eval 'keys $aref = 8; 1'\, 'keys $aref'; ok eval 'keys $href = 8; 1'\, 'keys $href';
On Mon Mar 21 20:40:19 2011\, sprout wrote:
On Mar 20\, 2011\, at 4:30 PM\, Jesse via RT wrote:
I'd sort of been waiting to see the testing of edge cases get sorted out.
I’m afraid I had forgotten about the edge cases. Now that I think about it though\, we have a chicken-and-egg problem. There is no problem with push\, splice\, etc.\, as the edge cases are all currently errors and can be resolved later without backward compatibility concerns. The edges cases for keys\, each and values\, however\, entirely depend upon your decision with regard to array refs and objects.
Attached is a first go at a test script. Edit the JC constant to choose between the different behaviours tested for.
BTW\, I noticed that ck_push is duplicating some logic from ck_fun. That was why in my first patch to this ticket I deleted ck_push and added some code to the array section of ck_fun.
On Mon\, Mar 21\, 2011 at 08:39:42PM -0700\, Father Chrysostomos wrote:
How plausible would it be to hide this feature behind a "use feature" directive for 5.14?
That would be possible\, but wouldn’t it be strange to put it there? What would the same ‘feature’ feature do in 5.16?
I agree. 'use feature' is inappropriate here.
I think marking that aspect of those functions as experimental in the docs (and removing the ‘or ARRAYREF’ from the =item directives in perlfunc) would be acceptable.
Agreed. I think the docs should make clear that any use apart from on an *unblessed* array or hash ref is undefined and highly subject to change.
Note also that traditionally we don't change such behaviours during maint cycles (the smartmatch in 5.10.1 being the big ugly exception).
I'm happy to make the doc changes if Jesse gives the nod.
-- Monto Blanco... scorchio!
On Mar 30\, 2011\, at 7:40 AM\, Dave Mitchell wrote:
On Mon\, Mar 21\, 2011 at 08:39:42PM -0700\, Father Chrysostomos wrote:
How plausible would it be to hide this feature behind a "use feature" directive for 5.14?
That would be possible\, but wouldn’t it be strange to put it there? What would the same ‘feature’ feature do in 5.16?
I agree. 'use feature' is inappropriate here.
I think marking that aspect of those functions as experimental in the docs (and removing the ‘or ARRAYREF’ from the =item directives in perlfunc) would be acceptable.
Agreed. I think the docs should make clear that any use apart from on an *unblessed* array or hash ref is undefined and highly subject to change.
For push\, shift\, etc.\, the only controversial cases are currently errors.
For keys\, each and values\, even unblessed array refs are still controversial. So I think the easiest approach for now is to mark keys/each/values with *any* scalar as being experimental and subject to change.
Attached is a diff with my recommendation as to how the docs should be changed.
Note also that traditionally we don't change such behaviours during maint cycles (the smartmatch in 5.10.1 being the big ugly exception).
I'm happy to make the doc changes if Jesse gives the nod.
After poring over the test scripts and various recommendations\, requests\, suggestions and concerns\, I believe that the right course of action at this point is what\, I believe was referred to as "proposal 2" at some point in the past:
* keys\, each and values should only magically dereference unblessed hashrefs and arrayrefs.
* keys\, each and values should behave as they did in 5.12 for blessed references
* For 5.14\, we will mark this behavior as experimental\, with an eye toward removing the experimental marker late in the 5.15 series based on positive user feedback.
I'm really sorry that I've been so avoidant on this one.
-Jesse
As happens with some regularity\, I made a pronouncement that was too vague and didn't cover all the cases I was being asked about.
Let me try again:
The following should magically dereference unblessed arrayrefs: push\, pop\, shift\, unshift
The following should magically dereference unblessed hashrefs or arrayrefs: keys\, values\, each
Everything else should work like it did in 5.12.
Now\, given that pronouncement\, I _believe_ we have some bugs to fix and some extra functionalty to remove.
-j
On Tue\, Apr 12\, 2011 at 06:50:03AM -0400\, Jesse Vincent wrote:
As happens with some regularity\, I made a pronouncement that was too vague and didn't cover all the cases I was being asked about.
Let me try again:
The following should magically dereference unblessed arrayrefs: push\, pop\, shift\, unshift
No splice?
The following should magically dereference unblessed hashrefs or arrayrefs: keys\, values\, each
Everything else should work like it did in 5.12.
Now\, given that pronouncement\, I _believe_ we have some bugs to fix and some extra functionalty to remove.
Abigail
On Tue\, Apr 12\, 2011 at 12:55:00PM +0200\, Abigail wrote:
On Tue\, Apr 12\, 2011 at 06:50:03AM -0400\, Jesse Vincent wrote:
As happens with some regularity\, I made a pronouncement that was too vague and didn't cover all the cases I was being asked about.
Let me try again:
The following should magically dereference unblessed arrayrefs: push\, pop\, shift\, unshift
No splice?
*sigh* The first version of this message had just said "keys and friends" Can I get away with falling back on "the customary set of functions that operate on hashes and arrays"?
On Tue\, Apr 12\, 2011 at 7:02 AM\, Jesse Vincent \jesse@​fsck\.com wrote:
On Tue\, Apr 12\, 2011 at 12:55:00PM +0200\, Abigail wrote:
On Tue\, Apr 12\, 2011 at 06:50:03AM -0400\, Jesse Vincent wrote:
As happens with some regularity\, I made a pronouncement that was too vague and didn't cover all the cases I was being asked about.
Let me try again:
The following should magically dereference unblessed arrayrefs: push\, pop\, shift\, unshift
No splice?
*sigh* The first version of this message had just said "keys and friends" Can I get away with falling back on "the customary set of functions that operate on hashes and arrays"?
Splice was not made magical like push/pop/etc.
-- David
On Tue\, Apr 12\, 2011 at 7:41 AM\, David Golden \xdaveg@​gmail\.com wrote:
On Tue\, Apr 12\, 2011 at 7:02 AM\, Jesse Vincent \jesse@​fsck\.com wrote:
On Tue\, Apr 12\, 2011 at 12:55:00PM +0200\, Abigail wrote:
On Tue\, Apr 12\, 2011 at 06:50:03AM -0400\, Jesse Vincent wrote:
As happens with some regularity\, I made a pronouncement that was too vague and didn't cover all the cases I was being asked about.
Let me try again:
The following should magically dereference unblessed arrayrefs: push\, pop\, shift\, unshift
No splice?
*sigh* The first version of this message had just said "keys and friends" Can I get away with falling back on "the customary set of functions that operate on hashes and arrays"?
Splice was not made magical like push/pop/etc.
Sorry -- yes it was and it should be made to work only on unblessed references like push/pop/shift/unshift.
Apologies for the confusion. I'm in week 3 of my sleep deprivation experiment and my memory occasionally misfires. :-)
-- David
On Apr 12\, 2011\, at 6:25 AM\, David Golden via RT wrote:
On Tue\, Apr 12\, 2011 at 7:41 AM\, David Golden \xdaveg@​gmail\.com wrote:
On Tue\, Apr 12\, 2011 at 7:02 AM\, Jesse Vincent \jesse@​fsck\.com wrote:
On Tue\, Apr 12\, 2011 at 12:55:00PM +0200\, Abigail wrote:
On Tue\, Apr 12\, 2011 at 06:50:03AM -0400\, Jesse Vincent wrote:
As happens with some regularity\, I made a pronouncement that was too vague and didn't cover all the cases I was being asked about.
Let me try again:
The following should magically dereference unblessed arrayrefs: push\, pop\, shift\, unshift
No splice?
*sigh* The first version of this message had just said "keys and friends" Can I get away with falling back on "the customary set of functions that operate on hashes and arrays"?
Splice was not made magical like push/pop/etc.
Sorry -- yes it was and it should be made to work only on unblessed references like push/pop/shift/unshift.
Apologies for the confusion. I'm in week 3 of my sleep deprivation experiment and my memory occasionally misfires. :-)
Jesse: What about keys %hashref as an lvalue?
David: Are you going to do this\, or shall I go ahead?
On Sun\, Apr 17\, 2011 at 8:08 PM\, Father Chrysostomos \sprout@​cpan\.org wrote:
Jesse: What about keys %hashref as an lvalue?
David: Are you going to do this\, or shall I go ahead?
Father C: I've got a 3 week old baby and family in town\, so I've hardly touched my computer. If you or someone could make the necessary changes for Jesses\, I'd greatly appreciate it.
Regards\, David
On Sun\, Apr 17\, 2011 at 05:08:38PM -0700\, Father Chrysostomos wrote:
Jesse: What about keys %hashref as an lvalue?
On an unblessed hashref\, my ideal would be for it to work as close to how keys would work on a hash. I suppose that yes\, the lvalue behavior should work the same way.
Thanks for the catch.
--
On Apr 17\, 2011\, at 6:36 PM\, David Golden wrote:
On Sun\, Apr 17\, 2011 at 8:08 PM\, Father Chrysostomos \sprout@​cpan\.org wrote:
Jesse: What about keys %hashref as an lvalue?
David: Are you going to do this\, or shall I go ahead?
Father C: I've got a 3 week old baby and family in town\, so I've hardly touched my computer. If you or someone could make the necessary changes for Jesses\, I'd greatly appreciate it.
OK\, I will.
sprout's series of commits just now gets us to the place we hoped to be for 5.14.0. I'm resolving this ticket.
@obra - Status changed from 'open' to 'resolved'
Migrated from rt.perl.org#80626 (status was 'resolved')
Searchable as RT80626$