Lexical subs

[p5pRT]

Migrated from rt.perl.org#113930 (status was 'resolved')

Searchable as RT113930$

[p5pRT]

From @cpansprout

There doesn’t appear to be a ticket for this already\, so I’m making one. There is also another reason​: I think I know how to make it work.

The last time this came up\, the two main issues were​:

1)

If ‘my sub’ follows the same scoping rules as existing my and sub declarations\, then the lexical sub will not be visible inside itself\, unless declared beforehand. But if ‘my’ creates a new pad entry\, then this won’t work​:

my sub foo; my sub foo { ... }

because the second one creates a new pad entry and does not modify the existing one.

2)

Should declarations be compile-time or run-time? If they are compile-time declarations\, then how does one redefine a sub (c.f. problem 1); and how does a lexical sub close over variables? If it is a run-time thing\, then it is radically different from ‘sub foo {...}’\, so shouldn’t it look different?

These questions can be summarized as​: When does a lexical closure close over outer variables? What else has to happen to make it work?

---

I think the solution to #1 is to make ‘sub foo { ... }’ respect a lexical declaration that precedes it. Just as $x = 3 does not assign to the package variable $x after a my $x declaration\, so ‘sub foo { ... }’ should not create a package sub after ‘my sub foo;’.

That in itself pretty much resolves #2\, but we have (at least) two variants​:

• Lexical subs close over variables when the name comes into scope (which could happen multiple times with loops). • Lexical subs close over variables when referenced or called. • Lexical subs close over variables when *first* referenced or called.

For the most part\, there is no observable difference here\, until it comes to referential identity and for loops.

I don’t think we have to worry about referential identity\, since optimisations would make all three variants nearly identical in that regard.

For for loops we have to decide what this will do​:

for my $x (1..10) {   my sub foo { push @​results\, $x }   foo(); }

The @​results will be 1..10\, 1..10 and (1)x10\, respectively for the three variants.

And what about this?

my sub foo; for my $x (1..10) {   sub foo { push @​results\, $x }   foo(); }

Results​: (undef)x10\, 1..10\, (1)x10

And this would produce the same​:

my $x; my sub foo { push @​results\, $x } for $x (1..10) {   foo(); }

In all these cases\, the second variant seems more intuitive\, at least to me. In fact\, that’s similar to how format closures work. This prints the numbers from 1 to 10​:

my $x; format = @​\<\<\<\<\<\< $x . for $x (1..10) {   write; }

If we introduce lexical aliases via \my $x = $y\, we will need to figure this out for the sake of that\, too.

---

Next issue​: What should state sub do? Should it behave the same way as my sub? Should it close over the variables the first time through the function? Should it be an error (currently it is not) until we decide?

I think it should be just a named package sub (‘Variable will not stay shared’ and all)\, but visible only in a lexical scope.

---

I think it is pretty obvious that our sub should create a lexical alias to a package sub\, so that this will work​:

sub foo { warn "main" } package bar; foo(); # calls main​::foo

---

‘my format’ and ‘my package’ would be going a bit too far. :-)

[p5pRT]

From @cpansprout

On Sun Jul 01 14​:23​:26 2012\, sprout wrote​:

Next issue​: What should state sub do? Should it behave the same way as my sub? Should it close over the variables the first time through the function? Should it be an error (currently it is not) until we decide?

I think it should be just a named package sub (‘Variable will not stay shared’ and all)\, but visible only in a lexical scope.

s/just/just like/

--

Father Chrysostomos

[p5pRT]

From [Unknown Contact. See original ticket]

On Sun Jul 01 14​:23​:26 2012\, sprout wrote​:

Next issue​: What should state sub do? Should it behave the same way as my sub? Should it close over the variables the first time through the function? Should it be an error (currently it is not) until we decide?

I think it should be just a named package sub (‘Variable will not stay shared’ and all)\, but visible only in a lexical scope.

s/just/just like/

--

Father Chrysostomos

[p5pRT]

@cpansprout - Status changed from 'new' to 'open'

[p5pRT]

From @kentfredric

On 2 July 2012 09​:23\, Father Chrysostomos \perlbug\-followup@&#8203;perl\.org wrote​:

for my $x (1..10) { my sub foo { push @​results\, $x } foo(); }

The @​results will be 1..10\, 1..10 and (1)x10\, respectively for the three variants.

As far as what I expect would happen\, I'd imagine\, if I saw that code\, that it would do the same as​:

for my $x (1..10) {   my $foo = sub { push @​results\, $x }   $foo(); }

And what about this?

my sub foo; for my $x (1..10) { sub foo { push @​results\, $x } foo(); }

If I saw that\, I'd expect the same as​:

my $foo = sub {}; for my $x (1..10) {   $foo = sub { push @​results\, $x };   $foo(); }

In both case\, my brain sees "my sub foo" and thinks its just sugar for "my $foo = sub "

However\, if you were to use 'state foo' \, the expectation I'd have is :

for my $x (1..10) {   state sub foo { push @​results\, $x }   foo(); }

would be the same as​:

my $foo; for my $x (1..10) {   $foo //= sub { push @​results\, $x }   $foo(); }

That being\, @​results would be (1)x10 ( Though\, slightly unintuitive I feel )

I think people who are doing that though are doing it to be "optimised" somehow\, and would expect the behaviour of​:

my $foo; my $x; for $x (1..10) {   $foo //= sub { push @​results\, $x }   $foo(); }

-- Kent

perl -e "print substr( \"edrgmaM SPA NOcomil.ic\\@​tfrken\"\, \$_ * 3\, 3 ) for ( 9\,8\,0\,7\,1\,6\,5\,4\,3\,2 );"

[p5pRT]

From @cpansprout

newMYSUB is listed in the API. (It has an A in embed.fnc and shows up in the undocumented list in perlapi.pod.)

This is its signature​:

#ifdef PERL_MAD OP * #else void #endif Perl_newMYSUB(pTHX_ I32 floor\, OP *o\, OP *proto\, OP *attrs\, OP *block)

Implementing lexical subs within the constrains of the current parameters and return value is not impossible\, but hardly sane. It would be easier to make newMYSUB a stub that dies\, and create a completely new function.

Instead\, can I just change this? Nothing uses it. All it does currently is die anyway.

--

Father Chrysostomos

[p5pRT]

From [Unknown Contact. See original ticket]

newMYSUB is listed in the API. (It has an A in embed.fnc and shows up in the undocumented list in perlapi.pod.)

This is its signature​:

#ifdef PERL_MAD OP * #else void #endif Perl_newMYSUB(pTHX_ I32 floor\, OP *o\, OP *proto\, OP *attrs\, OP *block)

Implementing lexical subs within the constrains of the current parameters and return value is not impossible\, but hardly sane. It would be easier to make newMYSUB a stub that dies\, and create a completely new function.

Instead\, can I just change this? Nothing uses it. All it does currently is die anyway.

--

Father Chrysostomos

[p5pRT]

From @ikegami

On Thu\, Jul 5\, 2012 at 5​:46 PM\, Father Chrysostomos via RT \< perlbug-comment@​perl.org> wrote​:

newMYSUB is listed in the API. (It has an A in embed.fnc and shows up in the undocumented list in perlapi.pod.)

This is its signature​:

#ifdef PERL_MAD OP * #else void #endif Perl_newMYSUB(pTHX_ I32 floor\, OP *o\, OP *proto\, OP *attrs\, OP *block)

Implementing lexical subs within the constrains of the current parameters and return value is not impossible\, but hardly sane. It would be easier to make newMYSUB a stub that dies\, and create a completely new function.

Instead\, can I just change this? Nothing uses it. All it does currently is die anyway.

I'm particularly interested in the ability of "use Module;" to create a lexical sub in the using scope. I looked at C\<\< newMYSUB >>\, but I ended up using Lexical​::Sub for Syntax​::Feature​::QwComments and :​:Loop. So if your new interface makes that easy\, I like it :D

- Eric

[p5pRT]

From @cpansprout

On Thu Jul 05 15​:11​:55 2012\, ikegami@​adaelis.com wrote​:

On Thu\, Jul 5\, 2012 at 5​:46 PM\, Father Chrysostomos via RT \< perlbug-comment@​perl.org> wrote​:

newMYSUB is listed in the API. (It has an A in embed.fnc and shows up in the undocumented list in perlapi.pod.)

This is its signature​:

#ifdef PERL_MAD OP * #else void #endif Perl_newMYSUB(pTHX_ I32 floor\, OP *o\, OP *proto\, OP *attrs\, OP *block)

Implementing lexical subs within the constrains of the current parameters and return value is not impossible\, but hardly sane. It would be easier to make newMYSUB a stub that dies\, and create a completely new function.

Instead\, can I just change this? Nothing uses it. All it does currently is die anyway.

I'm particularly interested in the ability of "use Module;" to create a lexical sub in the using scope. I looked at C\<\< newMYSUB >>\, but I ended up using Lexical​::Sub for Syntax​::Feature​::QwComments and :​:Loop. So if your new interface makes that easy\, I like it :D

If you already have a compiled CV\, it might be possible using PAD_SETSV(pad_add_name_pv("&foo"\, 0\, 0\, 0)\, cv). Currently in bleadperl you can do that (untested)\, but ‘foo’ in the source code won’t look it up.

My changes should make ‘foo’ generate a padcv op (nonexistent in blead) in that case.

newMYSUB is probably not what you want\, as it is analogous to newSUB​: It takes PL_compcv and stuffs it in the slot indicated by the second argument\, attaching the passed-in op tree\, and popping the stack back to what it was before start_subparse() (assuming you passed the return value of start_subparse() as the first argument to newSUB).

--

Father Chrysostomos

[p5pRT]

From [Unknown Contact. See original ticket]

On Thu Jul 05 15​:11​:55 2012\, ikegami@​adaelis.com wrote​:

On Thu\, Jul 5\, 2012 at 5​:46 PM\, Father Chrysostomos via RT \< perlbug-comment@​perl.org> wrote​:

newMYSUB is listed in the API. (It has an A in embed.fnc and shows up in the undocumented list in perlapi.pod.)

This is its signature​:

#ifdef PERL_MAD OP * #else void #endif Perl_newMYSUB(pTHX_ I32 floor\, OP *o\, OP *proto\, OP *attrs\, OP *block)

Implementing lexical subs within the constrains of the current parameters and return value is not impossible\, but hardly sane. It would be easier to make newMYSUB a stub that dies\, and create a completely new function.

Instead\, can I just change this? Nothing uses it. All it does currently is die anyway.

I'm particularly interested in the ability of "use Module;" to create a lexical sub in the using scope. I looked at C\<\< newMYSUB >>\, but I ended up using Lexical​::Sub for Syntax​::Feature​::QwComments and :​:Loop. So if your new interface makes that easy\, I like it :D

If you already have a compiled CV\, it might be possible using PAD_SETSV(pad_add_name_pv("&foo"\, 0\, 0\, 0)\, cv). Currently in bleadperl you can do that (untested)\, but ‘foo’ in the source code won’t look it up.

My changes should make ‘foo’ generate a padcv op (nonexistent in blead) in that case.

newMYSUB is probably not what you want\, as it is analogous to newSUB​: It takes PL_compcv and stuffs it in the slot indicated by the second argument\, attaching the passed-in op tree\, and popping the stack back to what it was before start_subparse() (assuming you passed the return value of start_subparse() as the first argument to newSUB).

--

Father Chrysostomos

[p5pRT]

From @doy

On Thu\, Jul 05\, 2012 at 02​:46​:38PM -0700\, Father Chrysostomos via RT wrote​:

newMYSUB is listed in the API. (It has an A in embed.fnc and shows up in the undocumented list in perlapi.pod.)

This is its signature​:

#ifdef PERL_MAD OP * #else void #endif Perl_newMYSUB(pTHX_ I32 floor\, OP *o\, OP *proto\, OP *attrs\, OP *block)

Implementing lexical subs within the constrains of the current parameters and return value is not impossible\, but hardly sane. It would be easier to make newMYSUB a stub that dies\, and create a completely new function.

Instead\, can I just change this? Nothing uses it. All it does currently is die anyway.

If all it does is die\, clearly nobody can be using this for any real purpose. I'd say go ahead and change it.

-doy

[p5pRT]

From @ikegami

On Thu\, Jul 5\, 2012 at 6​:59 PM\, Father Chrysostomos via RT \< perlbug-comment@​perl.org> wrote​:

I'm particularly interested in the ability of "use Module;" to create a lexical sub in the using scope. I looked at C\<\< newMYSUB >>\, but I ended up using Lexical​::Sub for Syntax​::Feature​::QwComments and :​:Loop. So if your new interface makes that easy\, I like it :D

If you already have a compiled CV\, it might be possible using PAD_SETSV(pad_add_name_pv("&foo"\, 0\, 0\, 0)\, cv). Currently in bleadperl you can do that (untested)\, but ‘foo’ in the source code won’t look it up.

No\, I did not have an already compiled CV. I needed a stub to pass to cv_set_call_parser and cv_set_call_checker In fact\, I found I just needed to *declare* the sub\, so I ended up simply using

CV* const qwcv = get_cvn_flags("Syntax​::Feature​::QwComments​::replacement_qw"\, 43\, GV_ADD);

If you already have a compiled CV\, it might be possible using

PAD_SETSV(pad_add_name_pv("&foo"\, 0\, 0\, 0)\, cv). Currently in bleadperl you can do that (untested)\, but ‘foo’ in the source code won’t look it up.

Are you planning on having Perl look there?

- Eric

[p5pRT]

From @cpansprout

On Thu Jul 05 20​:56​:18 2012\, ikegami@​adaelis.com wrote​:

On Thu\, Jul 5\, 2012 at 6​:59 PM\, Father Chrysostomos via RT \< perlbug-comment@​perl.org> wrote​:

I'm particularly interested in the ability of "use Module;" to create a lexical sub in the using scope. I looked at C\<\< newMYSUB >>\, but I ended up using Lexical​::Sub for Syntax​::Feature​::QwComments and :​:Loop. So if your new interface makes that easy\, I like it :D

If you already have a compiled CV\, it might be possible using PAD_SETSV(pad_add_name_pv("&foo"\, 0\, 0\, 0)\, cv).

No\, I did not have an already compiled CV. I needed a stub to pass to cv_set_call_parser and cv_set_call_checker In fact\, I found I just needed to *declare* the sub\, so I ended up simply using

CV* const qwcv = get_cvn_flags("Syntax​::Feature​::QwComments​::replacement_qw"\, 43\, GV_ADD);

I misspoke. I meant ‘your own CV’\, as opposed to PL_compcv.

Currently in bleadperl you can do that (untested)\, but ‘foo’ in the source code won’t look it up.

Are you planning on having Perl look there?

Yes. On the sprout/lexsub branch it already does\, but only works for our subs.

--

Father Chrysostomos

[p5pRT]

From [Unknown Contact. See original ticket]

On Thu Jul 05 20​:56​:18 2012\, ikegami@​adaelis.com wrote​:

On Thu\, Jul 5\, 2012 at 6​:59 PM\, Father Chrysostomos via RT \< perlbug-comment@​perl.org> wrote​:

I'm particularly interested in the ability of "use Module;" to create a lexical sub in the using scope. I looked at C\<\< newMYSUB >>\, but I ended up using Lexical​::Sub for Syntax​::Feature​::QwComments and :​:Loop. So if your new interface makes that easy\, I like it :D

If you already have a compiled CV\, it might be possible using PAD_SETSV(pad_add_name_pv("&foo"\, 0\, 0\, 0)\, cv).

No\, I did not have an already compiled CV. I needed a stub to pass to cv_set_call_parser and cv_set_call_checker In fact\, I found I just needed to *declare* the sub\, so I ended up simply using

CV* const qwcv = get_cvn_flags("Syntax​::Feature​::QwComments​::replacement_qw"\, 43\, GV_ADD);

I misspoke. I meant ‘your own CV’\, as opposed to PL_compcv.

Currently in bleadperl you can do that (untested)\, but ‘foo’ in the source code won’t look it up.

Are you planning on having Perl look there?

Yes. On the sprout/lexsub branch it already does\, but only works for our subs.

--

Father Chrysostomos

[p5pRT]

From @cpansprout

On Sun Jul 01 14​:23​:26 2012\, sprout wrote​:

...[W]e have (at least) two variants​:

• Lexical subs close over variables when the name comes into scope (which could happen multiple times with loops). • Lexical subs close over variables when referenced or called. • Lexical subs close over variables when *first* referenced or called.

For the most part\, there is no observable difference here\, until it comes to referential identity and for loops.

I don’t think we have to worry about referential identity\, since optimisations would make all three variants nearly identical in that regard.

Actually we do.

State variables are not persistent across anonymous subroutines.

  push @​subs\, sub { state $x } for 1..10;

will give you ten subroutines\, each with a different $x.

I find that to be completely counterintuitive. (It is also completely undocumented.) It means that ‘state $x’ might only create one variable\, or might create more than one\, depending on what type of sub it is defined in.

It also creates yet another set of scoping rules for people to remember. Flip-flops are shared between clones\, so why not state variables?

The problem it raises for lexical subs is that it is not at all clear when my sub foo { state $x } will create a new $x.

If state variables were intended to replace this sort of thing​:

{   my $x;   sub fooer {   sub { ++$x; foo($x); }   } }

then they have failed to do so.

Is this something we can fix\, or will it break too many things?

--

Father Chrysostomos

[p5pRT]

From [Unknown Contact. See original ticket]

On Sun Jul 01 14​:23​:26 2012\, sprout wrote​:

...[W]e have (at least) two variants​:

• Lexical subs close over variables when the name comes into scope (which could happen multiple times with loops). • Lexical subs close over variables when referenced or called. • Lexical subs close over variables when *first* referenced or called.

For the most part\, there is no observable difference here\, until it comes to referential identity and for loops.

I don’t think we have to worry about referential identity\, since optimisations would make all three variants nearly identical in that regard.

Actually we do.

State variables are not persistent across anonymous subroutines.

  push @​subs\, sub { state $x } for 1..10;

will give you ten subroutines\, each with a different $x.

I find that to be completely counterintuitive. (It is also completely undocumented.) It means that ‘state $x’ might only create one variable\, or might create more than one\, depending on what type of sub it is defined in.

It also creates yet another set of scoping rules for people to remember. Flip-flops are shared between clones\, so why not state variables?

The problem it raises for lexical subs is that it is not at all clear when my sub foo { state $x } will create a new $x.

If state variables were intended to replace this sort of thing​:

{   my $x;   sub fooer {   sub { ++$x; foo($x); }   } }

then they have failed to do so.

Is this something we can fix\, or will it break too many things?

--

Father Chrysostomos

[p5pRT]

From perl-diddler@tlinx.org

` Father Chrysostomos via RT wrote​:

On Sun Jul 01 14​:23​:26 2012\, sprout wrote​:

...[W]e have (at least) two variants​:

• Lexical subs close over variables when the name comes into scope (which could happen multiple times with loops). • Lexical subs close over variables when referenced or called. • Lexical subs close over variables when *first* referenced or called.

For the most part\, there is no observable difference here\, until it comes to referential identity and for loops.

I don’t think we have to worry about referential identity\, since optimisations would make all three variants nearly identical in that regard.

Actually we do.

State variables are not persistent across anonymous subroutines.

push @​subs\, sub { state $x } for 1..10;

That hurts.

I would have thought it would have declared it once like a 'my' and named sub...

Erg......Honestly\, since 'my' already isn't persistent across such\, is it possible to fix state?

will give you ten subroutines\, each with a different $x.

I find that to be completely counterintuitive. (It is also completely undocumented.) It means that ‘state $x’ might only create one variable\, or might create more than one\, depending on what type of sub it is defined in.

It also creates yet another set of scoping rules for people to remember. Flip-flops are shared between clones\, so why not state variables?

The problem it raises for lexical subs is that it is not at all clear when my sub foo { state $x } will create a new $x.

If state variables were intended to replace this sort of thing​:

{ my $x; sub fooer { sub { ++$x; foo($x); } } }

then they have failed to do so.

Is this something we can fix\, or will it break too many things?

[p5pRT]

From @cpansprout

On Sat Jul 07 11​:35​:48 2012\, sprout wrote​:

On Sun Jul 01 14​:23​:26 2012\, sprout wrote​:

...[W]e have (at least) two variants​:

• Lexical subs close over variables when the name comes into scope (which could happen multiple times with loops). • Lexical subs close over variables when referenced or called. • Lexical subs close over variables when *first* referenced or called.

For the most part\, there is no observable difference here\, until it comes to referential identity and for loops.

I don’t think we have to worry about referential identity\, since optimisations would make all three variants nearly identical in that regard.

Actually we do.

State variables are not persistent across anonymous subroutines.

push @​subs\, sub { state $x } for 1..10;

will give you ten subroutines\, each with a different $x.

I find that to be completely counterintuitive. (It is also completely undocumented.) It means that ‘state $x’ might only create one variable\, or might create more than one\, depending on what type of sub it is defined in.

It also creates yet another set of scoping rules for people to remember. Flip-flops are shared between clones\, so why not state variables?

The problem it raises for lexical subs is that it is not at all clear when my sub foo { state $x } will create a new $x.

If state variables were intended to replace this sort of thing​:

{ my $x; sub fooer { sub { ++$x; foo($x); } } }

then they have failed to do so.

Is this something we can fix\, or will it break too many things?

In fact\, they have been this way since they were added in commit 952306aca1\, at least for closures. The code that causes state vars to be created anew was not touched by that patch\, so it seems like an oversight. Other anonymous subs started recreating state vars in commit a74073ad\, sixteen months later\, which I think was for consistency’s sake.

...

And I just found this message\, which shows that it was very deliberate​:

http​://www.nntp.perl.org/group/perl.perl5.porters/;msgid=20070906085723.GA30617@​iabyn.com

So now I’m stuck on how to implement my sub foo { state $x }.

--

Father Chrysostomos

[p5pRT]

From [Unknown Contact. See original ticket]

On Sat Jul 07 11​:35​:48 2012\, sprout wrote​:

On Sun Jul 01 14​:23​:26 2012\, sprout wrote​:

...[W]e have (at least) two variants​:

• Lexical subs close over variables when the name comes into scope (which could happen multiple times with loops). • Lexical subs close over variables when referenced or called. • Lexical subs close over variables when *first* referenced or called.

For the most part\, there is no observable difference here\, until it comes to referential identity and for loops.

I don’t think we have to worry about referential identity\, since optimisations would make all three variants nearly identical in that regard.

Actually we do.

State variables are not persistent across anonymous subroutines.

push @​subs\, sub { state $x } for 1..10;

will give you ten subroutines\, each with a different $x.

I find that to be completely counterintuitive. (It is also completely undocumented.) It means that ‘state $x’ might only create one variable\, or might create more than one\, depending on what type of sub it is defined in.

It also creates yet another set of scoping rules for people to remember. Flip-flops are shared between clones\, so why not state variables?

The problem it raises for lexical subs is that it is not at all clear when my sub foo { state $x } will create a new $x.

If state variables were intended to replace this sort of thing​:

{ my $x; sub fooer { sub { ++$x; foo($x); } } }

then they have failed to do so.

Is this something we can fix\, or will it break too many things?

In fact\, they have been this way since they were added in commit 952306aca1\, at least for closures. The code that causes state vars to be created anew was not touched by that patch\, so it seems like an oversight. Other anonymous subs started recreating state vars in commit a74073ad\, sixteen months later\, which I think was for consistency’s sake.

...

And I just found this message\, which shows that it was very deliberate​:

http​://www.nntp.perl.org/group/perl.perl5.porters/;msgid=20070906085723.GA30617@​iabyn.com

So now I’m stuck on how to implement my sub foo { state $x }.

--

Father Chrysostomos

[p5pRT]

From tchrist@perl.com

"Father Chrysostomos via RT" \perlbug\-comment@&#8203;perl\.org wrote   on Sat\, 07 Jul 2012 11​:35​:49 PDT​:

On Sun Jul 01 14​:23​:26 2012\, sprout wrote​:

...[W]e have (at least) two variants​:

• Lexical subs close over variables when the name comes into scope (which could happen multiple times with loops). • Lexical subs close over variables when referenced or called. • Lexical subs close over variables when *first* referenced or called.

For the most part\, there is no observable difference here\, until it comes to referential identity and for loops.

I don’t think we have to worry about referential identity\, since optimisations would make all three variants nearly identical in that regard.

Actually we do.

State variables are not persistent across anonymous subroutines.

push @​subs\, sub { state $x } for 1..10;

will give you ten subroutines\, each with a different $x.

I find that to be completely counterintuitive. (It is also completely undocumented.)

Not so. Camel v4\, chapter 7\, page 324​:

  Finally\, when we say that a state variable is initialized only once\, we don’t mean   to imply that state variables in separate closures are the same variables. They   aren’t\, so each gets its own initialization. This is how state variables differ from   static variables in other languages.

Larry was quite certain about that part\, because he wanted to make sure people understood how it worked\, and that it did so by deliberate intent​: each closure is meant to get *its own* copy of a state variable.

It means that ‘state $x’ might only create one variable\, or might create more than one\, depending on what type of sub it is defined in.

True.

It also creates yet another set of scoping rules for people to remember. Flip-flops are shared between clones\, so why not state variables?

Because it was designed to work this way.

The problem it raises for lexical subs is that it is not at all clear when my sub foo { state $x } will create a new $x.

If we view C\ as a declaration\, than I cannot see why there would ever be more than one of them\, and therefore\, more than one $x.

Are you worried about this situation?

  sub outer {   my @​subs;   for $i (1 .. 10) {   my sub inner {   state $x = rand();   return [ $i => $x ];   }   push @​subs\, \&inner;   }   return @​subs;   }

The important question here is whether all those $x variables have the same random number\, or whether they have different ones.

And no\, I don't know what the right answer is. I do agree that is the right question\, though. :)

If state variables were intended to replace this sort of thing​:

{ my $x; sub fooer { sub { ++$x; foo($x); } } }

then they have failed to do so.

I don't believe they were.

Is this something we can fix\, or will it break too many things?

I'm not sure "fix" is the operative term here\, considering that it would certainly break existing code.

  my @​subs;   for my $decade (0 .. 10) {   push @​subs\, sub {   state $counter = 10 * $decade;   return $counter++ % 10;   };   }

There's also the argument that breaking this changes the designed intent of state. And the documentation.

Can't see a good argument for all that.

Nonetheless\, I can see your conundrum. I don't suppose you have a current rakudo/perl6 and could check out how/whether it works there? S04 has this​:

  http​://perlcabal.org/syn/S04.html

  There is a new state declarator that introduces a lexically scoped   variable like my does\, but with a lifetime that persists for the life   of the closure\, so that it keeps its value from the end of one call to   the beginning of the next. Separate clones of the closure get separate   state variables. However\, recursive calls to the same clone use the   same state variable.

And later on​:

  The semantics of INIT and START are not equivalent to each other in the   case of cloned closures. An INIT only runs once for all copies of a cloned   closure. A START runs separately for each clone\, so separate clones can   keep separate state variables​:

  our $i = 0;   ...   $func = { state $x will start { $x = $i++ }; dostuff($i) };

  But state automatically applies "start" semantics to any initializer\, so this also works​:

  $func = { state $x = $i++; dostuff($i) }

  Each subsequent clone gets an initial state that is one higher than the   previous\, and each clone maintains its own state of $x\, because that's   what state variables do.

And later on there is this\, which is interesting but not completely revealing\, since it deals with a my not a state​:

  Lexical names do not share this problem\, since the symbol goes out of   scope synchronously with its usage. Unlike global subs\, they do not need a   compile-time binding\, but like global subs\, they perform a binding to the   lexical symbol at clone time (again\, conceptually at the entry to the   outer lexical scope\, but possibly deferred.)

  sub foo {   # conceptual cloning happens to both blocks below   my $x = 1;   my sub bar { print $x } # already conceptually cloned\, but can be lazily deferred   my &baz := { bar(); print $x }; # block is cloned immediately\, forcing cloning of bar   my $code = &bar; # this would also force bar to be cloned   return &baz;   }

There may also be applicable points in S06\, since you can my subs in perl6\, and in fact\, the default is my sub not our sub​:

  http​://perlcabal.org/syn/S06.html#Named_subroutines

Hm\, have you thought about our sub? I think it is the same in perl6 as regular perl5 subs are — that is\, ones interred in the package symbol table — but I might be wrong.

--tom

[p5pRT]

From @cpansprout

On Sat Jul 07 12​:53​:32 2012\, tom christiansen wrote​:

"Father Chrysostomos via RT" \perlbug\-comment@&#8203;perl\.org wrote on Sat\, 07 Jul 2012 11​:35​:49 PDT​:

On Sun Jul 01 14​:23​:26 2012\, sprout wrote​:

...[W]e have (at least) two variants​:

• Lexical subs close over variables when the name comes into scope (which could happen multiple times with loops). • Lexical subs close over variables when referenced or called. • Lexical subs close over variables when *first* referenced or called.

For the most part\, there is no observable difference here\, until it comes to referential identity and for loops.

I don’t think we have to worry about referential identity\, since optimisations would make all three variants nearly identical in that regard.

Actually we do.

State variables are not persistent across anonymous subroutines.

push @​subs\, sub { state $x } for 1..10;

will give you ten subroutines\, each with a different $x.

I find that to be completely counterintuitive. (It is also completely undocumented.)

Not so. Camel v4\, chapter 7\, page 324​:

Finally\, when we say that a state variable is initialized only

once\, we don’t mean to imply that state variables in separate closures are the same variables. They aren’t\, so each gets its own initialization. This is how state variables differ from static variables in other languages.

Larry was quite certain about that part\, because he wanted to make sure people understood how it worked\, and that it did so by deliberate intent​: each closure is meant to get *its own* copy of a state variable.

Thank you.

Could you write a patch to perlsub then? :-)

What it currently says is terribly vague. In fact\, I would go so far as to say that it doesn’t even state that ‘state’ creates persistent variables. It begins by saying that ‘state’ can be used instead of ‘my’ to declare a variable\, and then goes straight into examples. Examples are supposed to supplement\, not supplant\, descriptions.

It means that ‘state $x’ might only create one variable\, or might create more than one\, depending on what type of sub it is defined in.

True.

It also creates yet another set of scoping rules for people to remember. Flip-flops are shared between clones\, so why not state variables?

Because it was designed to work this way.

The problem it raises for lexical subs is that it is not at all clear when my sub foo { state $x } will create a new $x.

If we view C\ as a declaration\, than I cannot see why there would ever be more than one of them\, and therefore\, more than one $x.

Are you worried about this situation?

sub outer \{
    my @​subs;
    for $i \(1 \.\. 10\) \{
        my sub inner \{
            state $x = rand\(\);
            return \[ $i => $x \];
        \}
        push @​subs\, \\&inner;
    \}
    return @​subs;
\}

The important question here is whether all those $x variables have the same random number\, or whether they have different ones.

And no\, I don't know what the right answer is. I do agree that is the right question\, though. :)

I think I have the right answer now. If we document that only *anonymous* subroutines get their own copies of state variables when cloned\, then my subs (personal subs? idiotic subs?) share them.

In the example you gave\, those $x variables would all have the same random number.

If whether \&inner will clone the sub or not is supposed to be a matter of optimisation\, that’s the only way it can work.

There's also the argument that breaking this changes the designed intent of state. And the documentation.

Maybe the Camel\, but certainly not perlsub. :-)

And later on there is this\, which is interesting but not completely revealing\, since it deals with a my not a state​:

Lexical names do not share this problem\, since the symbol goes out

of scope synchronously with its usage. Unlike global subs\, they do not need a compile-time binding\, but like global subs\, they perform a binding to the lexical symbol at clone time (again\, conceptually at the entry to the outer lexical scope\, but possibly deferred.)

    sub foo \{
        \# conceptual cloning happens to both blocks below
        my $x = 1;
        my sub bar \{ print $x \}         \# already conceptually

cloned\, but can be lazily deferred my &baz := { bar(); print $x }; # block is cloned immediately\, forcing cloning of bar my $code = &bar; # this would also force bar to be cloned return &baz; }

That is interesting\, since it is very similar to what I came up with on my own.

If a sub is conceptually cloned when the block enters\, does that mean that my $code = &bar (\&bar in p5) twice in a row should produce the same value?

How does Perl 6 deal with my subs in for loops?

There may also be applicable points in S06\, since you can my subs in perl6\, and in fact\, the default is my sub not our sub​:

http​://perlcabal\.org/syn/S06\.html\#Named\_subroutines

Hm\, have you thought about our sub?

I’ve already implemented it. :-) See the sprout/lexsub branch.

It is analogous to our $var\, in that the declaration persists across package declarations.

$ ./perl -Ilib -e 'our sub foo { warn 42 } package bar; foo()' 42 at -e line 1.

$ ./perl -Ilib -MO=Concise -e 'package a; our sub foo { warn 42 } package bar; foo()' 6 \<@​> leave[1 ref] vKP/REFC ->(end) 1 \<0> enter ->2 2 \<;> nextstate(bar 3 -e​:1) v​:{ ->3 5 \<1> entersub[t2] vKS/TARG ->6 - \<1> ex-list K ->5 3 \<0> pushmark s ->4 - \<1> ex-rv2cv sK ->- 4 \<$> gv(*a​::foo) s ->5 -e syntax OK

I think it is the same in perl6 as regular perl5 subs are — that is\, ones interred in the package symbol table — but I might be wrong.

Whether it is or no\, I wot not. I have not followed Perl 6 developement closely at all. If it is different in Perl 6\, I do not advocate adopting ‘our sub’ from Perl 6\, since ‘our’ already works a certain way in Perl 5\, so ‘our sub’ should follow that.

--

Father Chrysostomos

[p5pRT]

From [Unknown Contact. See original ticket]

On Sat Jul 07 12​:53​:32 2012\, tom christiansen wrote​:

"Father Chrysostomos via RT" \perlbug\-comment@&#8203;perl\.org wrote on Sat\, 07 Jul 2012 11​:35​:49 PDT​:

On Sun Jul 01 14​:23​:26 2012\, sprout wrote​:

...[W]e have (at least) two variants​:

• Lexical subs close over variables when the name comes into scope (which could happen multiple times with loops). • Lexical subs close over variables when referenced or called. • Lexical subs close over variables when *first* referenced or called.

For the most part\, there is no observable difference here\, until it comes to referential identity and for loops.

I don’t think we have to worry about referential identity\, since optimisations would make all three variants nearly identical in that regard.

Actually we do.

State variables are not persistent across anonymous subroutines.

push @​subs\, sub { state $x } for 1..10;

will give you ten subroutines\, each with a different $x.

I find that to be completely counterintuitive. (It is also completely undocumented.)

Not so. Camel v4\, chapter 7\, page 324​:

Finally\, when we say that a state variable is initialized only

once\, we don’t mean to imply that state variables in separate closures are the same variables. They aren’t\, so each gets its own initialization. This is how state variables differ from static variables in other languages.

Larry was quite certain about that part\, because he wanted to make sure people understood how it worked\, and that it did so by deliberate intent​: each closure is meant to get *its own* copy of a state variable.

Thank you.

Could you write a patch to perlsub then? :-)

What it currently says is terribly vague. In fact\, I would go so far as to say that it doesn’t even state that ‘state’ creates persistent variables. It begins by saying that ‘state’ can be used instead of ‘my’ to declare a variable\, and then goes straight into examples. Examples are supposed to supplement\, not supplant\, descriptions.

It means that ‘state $x’ might only create one variable\, or might create more than one\, depending on what type of sub it is defined in.

True.

It also creates yet another set of scoping rules for people to remember. Flip-flops are shared between clones\, so why not state variables?

Because it was designed to work this way.

The problem it raises for lexical subs is that it is not at all clear when my sub foo { state $x } will create a new $x.

If we view C\ as a declaration\, than I cannot see why there would ever be more than one of them\, and therefore\, more than one $x.

Are you worried about this situation?

sub outer \{
    my @​subs;
    for $i \(1 \.\. 10\) \{
        my sub inner \{
            state $x = rand\(\);
            return \[ $i => $x \];
        \}
        push @​subs\, \\&inner;
    \}
    return @​subs;
\}

The important question here is whether all those $x variables have the same random number\, or whether they have different ones.

And no\, I don't know what the right answer is. I do agree that is the right question\, though. :)

I think I have the right answer now. If we document that only *anonymous* subroutines get their own copies of state variables when cloned\, then my subs (personal subs? idiotic subs?) share them.

In the example you gave\, those $x variables would all have the same random number.

If whether \&inner will clone the sub or not is supposed to be a matter of optimisation\, that’s the only way it can work.

There's also the argument that breaking this changes the designed intent of state. And the documentation.

Maybe the Camel\, but certainly not perlsub. :-)

And later on there is this\, which is interesting but not completely revealing\, since it deals with a my not a state​:

Lexical names do not share this problem\, since the symbol goes out

of scope synchronously with its usage. Unlike global subs\, they do not need a compile-time binding\, but like global subs\, they perform a binding to the lexical symbol at clone time (again\, conceptually at the entry to the outer lexical scope\, but possibly deferred.)

    sub foo \{
        \# conceptual cloning happens to both blocks below
        my $x = 1;
        my sub bar \{ print $x \}         \# already conceptually

cloned\, but can be lazily deferred my &baz := { bar(); print $x }; # block is cloned immediately\, forcing cloning of bar my $code = &bar; # this would also force bar to be cloned return &baz; }

That is interesting\, since it is very similar to what I came up with on my own.

If a sub is conceptually cloned when the block enters\, does that mean that my $code = &bar (\&bar in p5) twice in a row should produce the same value?

How does Perl 6 deal with my subs in for loops?

There may also be applicable points in S06\, since you can my subs in perl6\, and in fact\, the default is my sub not our sub​:

http​://perlcabal\.org/syn/S06\.html\#Named\_subroutines

Hm\, have you thought about our sub?

I’ve already implemented it. :-) See the sprout/lexsub branch.

It is analogous to our $var\, in that the declaration persists across package declarations.

$ ./perl -Ilib -e 'our sub foo { warn 42 } package bar; foo()' 42 at -e line 1.

$ ./perl -Ilib -MO=Concise -e 'package a; our sub foo { warn 42 } package bar; foo()' 6 \<@​> leave[1 ref] vKP/REFC ->(end) 1 \<0> enter ->2 2 \<;> nextstate(bar 3 -e​:1) v​:{ ->3 5 \<1> entersub[t2] vKS/TARG ->6 - \<1> ex-list K ->5 3 \<0> pushmark s ->4 - \<1> ex-rv2cv sK ->- 4 \<$> gv(*a​::foo) s ->5 -e syntax OK

I think it is the same in perl6 as regular perl5 subs are — that is\, ones interred in the package symbol table — but I might be wrong.

Whether it is or no\, I wot not. I have not followed Perl 6 developement closely at all. If it is different in Perl 6\, I do not advocate adopting ‘our sub’ from Perl 6\, since ‘our’ already works a certain way in Perl 5\, so ‘our sub’ should follow that.

--

Father Chrysostomos

[p5pRT]

From @cpansprout

I’m forwarding this to the Perl 6 language list\, so see if I can find an answer there.

[This conversation is about how lexical subs should be implemented in Perl 5. What Perl 6 does may help in determining how to iron out the edge cases.]

On Sat Jul 07 13​:23​:17 2012\, sprout wrote​:

On Sat Jul 07 12​:53​:32 2012\, tom christiansen wrote​:

Are you worried about this situation?

sub outer \{
    my @​subs;
    for $i \(1 \.\. 10\) \{
        my sub inner \{
            state $x = rand\(\);
            return \[ $i => $x \];
        \}
        push @​subs\, \\&inner;
    \}
    return @​subs;
\}

The important question here is whether all those $x variables have the same random number\, or whether they have different ones.

And no\, I don't know what the right answer is. I do agree that is the right question\, though. :)

I think I have the right answer now. If we document that only *anonymous* subroutines get their own copies of state variables when cloned\, then my subs (personal subs? idiotic subs?) share them.

In the example you gave\, those $x variables would all have the same random number.

If whether \&inner will clone the sub or not is supposed to be a matter of optimisation\, that’s the only way it can work. ...

And later on there is this\, which is interesting but not completely revealing\, since it deals with a my not a state​:

Lexical names do not share this problem\, since the symbol goes out

of scope synchronously with its usage. Unlike global subs\, they do not need a compile-time binding\, but like global subs\, they perform a binding to the lexical symbol at clone time (again\, conceptually at the entry to the outer lexical scope\, but possibly deferred.)

    sub foo \{
        \# conceptual cloning happens to both blocks below
        my $x = 1;
        my sub bar \{ print $x \}         \# already conceptually

cloned\, but can be lazily deferred my &baz := { bar(); print $x }; # block is cloned immediately\, forcing cloning of bar my $code = &bar; # this would also force bar to be cloned return &baz; }

That is interesting\, since it is very similar to what I came up with on my own.

If a sub is conceptually cloned when the block enters\, does that mean that my $code = &bar (\&bar in p5) twice in a row should produce the same value?

How does Perl 6 deal with my subs in for loops?

This question might be more appropriate​: In this example\, which @​a does the bar subroutine see (in Perl 6)?

  sub foo {   my @​a = (1\,2\,3);   my sub bar { say @​a };   @​a := [4\,5\,6];   bar();   }

--

Father Chrysostomos

[p5pRT]

From [Unknown Contact. See original ticket]

I’m forwarding this to the Perl 6 language list\, so see if I can find an answer there.

[This conversation is about how lexical subs should be implemented in Perl 5. What Perl 6 does may help in determining how to iron out the edge cases.]

On Sat Jul 07 13​:23​:17 2012\, sprout wrote​:

On Sat Jul 07 12​:53​:32 2012\, tom christiansen wrote​:

Are you worried about this situation?

sub outer \{
    my @​subs;
    for $i \(1 \.\. 10\) \{
        my sub inner \{
            state $x = rand\(\);
            return \[ $i => $x \];
        \}
        push @​subs\, \\&inner;
    \}
    return @​subs;
\}

The important question here is whether all those $x variables have the same random number\, or whether they have different ones.

And no\, I don't know what the right answer is. I do agree that is the right question\, though. :)

I think I have the right answer now. If we document that only *anonymous* subroutines get their own copies of state variables when cloned\, then my subs (personal subs? idiotic subs?) share them.

In the example you gave\, those $x variables would all have the same random number.

If whether \&inner will clone the sub or not is supposed to be a matter of optimisation\, that’s the only way it can work. ...

And later on there is this\, which is interesting but not completely revealing\, since it deals with a my not a state​:

Lexical names do not share this problem\, since the symbol goes out

of scope synchronously with its usage. Unlike global subs\, they do not need a compile-time binding\, but like global subs\, they perform a binding to the lexical symbol at clone time (again\, conceptually at the entry to the outer lexical scope\, but possibly deferred.)

    sub foo \{
        \# conceptual cloning happens to both blocks below
        my $x = 1;
        my sub bar \{ print $x \}         \# already conceptually

cloned\, but can be lazily deferred my &baz := { bar(); print $x }; # block is cloned immediately\, forcing cloning of bar my $code = &bar; # this would also force bar to be cloned return &baz; }

That is interesting\, since it is very similar to what I came up with on my own.

If a sub is conceptually cloned when the block enters\, does that mean that my $code = &bar (\&bar in p5) twice in a row should produce the same value?

How does Perl 6 deal with my subs in for loops?

This question might be more appropriate​: In this example\, which @​a does the bar subroutine see (in Perl 6)?

  sub foo {   my @​a = (1\,2\,3);   my sub bar { say @​a };   @​a := [4\,5\,6];   bar();   }

--

Father Chrysostomos

[p5pRT]

From tchrist@perl.com

"Father Chrysostomos via RT" \perlbug\-comment@&#8203;perl\.org wrote   on Sat\, 07 Jul 2012 17​:44​:46 PDT​:

I’m forwarding this to the Perl 6 language list\, so see if I can find an answer there.

I do have an answer from Damian\, which I will enclose below\, and a Rakudo result for you.

[This conversation is about how lexical subs should be implemented in Perl 5. What Perl 6 does may help in determining how to iron out the edge cases.]

[...]

This question might be more appropriate​: In this example\, which @​a does the bar subroutine see (in Perl 6)?

sub foo \{
    my @​a = \(1\,2\,3\);
    my sub bar \{ say @​a \};
    @​a := \[4\,5\,6\];
    bar\(\);
\}

The answer to your immediate question is that if you call foo()\, it prints out 456 under Rakudo.

Following is Damian's answer to my question\, shared with permission.

--tom

  From​: Damian Conway \damian@&#8203;conway\.org   To​: Tom Christiansen \tchrist@&#8203;perl\.com   CC​: Larry Wall \larry@&#8203;wall\.org   Date​: Sun\, 08 Jul 2012 07​:17​:19 +1000   Delivery-Date​: Sat\, 07 Jul 2012 15​:19​:09   Subject​: Re​: my subs and state vars   In-Reply-To​: \22255\.1341691089@&#8203;chthon

  X-Spam-Status​: No\, score=-102.6 required=4.5 tests=BAYES_00\,RCVD_IN_DNSWL_LOW\,   USER_IN_WHITELIST autolearn=ham version=3.3.0

  X-Google-Sender-Auth​: UHLwfgo2kyvv2prdl6qJm-RfLF8   Content-Type​: text/plain; charset=ISO-8859-1

  > It looks like perl5 may be close to having my subs\, but a puzzle   > has emerged about how in some circumstances to treat state   > variables within those. [I'm pretty sure that perl6 has thought   > this through thoroughly\, but [I] am personally unfamiliar with the   > outcome of said contemplations.]   >   > I bet you aren't\, though. Any ideas or clues?

  The right things to do (and what Rakudo actually does) is to treat   lexical subs as lexically scoped *instances* of the specified sub   within the current surrounding block.

  That is​: a lexical sub is like a "my" var\, in that you get a new one   each time the surrounding block is executed. Rather than like an "our"   variable\, where you get a new lexically scoped alias to the same package   scoped variable.

  By that reasoning\, state vars inside a my sub must belong to each   instance of the sub\, just as state vars inside anonymous subs belong to   each instance of the anonymous sub.

  Another way of thinking about what Perl 6 does is that​:

  my sub foo { whatever() }

  is just syntactic sugar for​:

  my &foo := sub { whatever() }

  That is​: create a lexically scoped Code object and alias it at run-time   to an anonymous subroutine. So the rules for state variables inside   lexical subs *must* be the same as the rules for state variables inside   anonymous subs\, since they're actually just two ways of creating the   same thing.

  With this approach\, in Perl 6 it's easy to specify exactly what you want​:

  sub recount_from ($n) {

  my sub counter {   state $count = $n; # Each instance of &counter has its own count   say $count--;   die if $count == 0;   }

  while prompt "recount $n> " {   counter;   }   }

  vs​:

  sub first_count_down_from ($n) {

  state $count = $n; # All instances of &counter share a common count

  my sub counter {   say $count--;   die if $count == 0;   }

  while prompt "first count $n> " {   counter;   }   }

  Feel free to forward the above to anyone who might find it useful.

  Damian

[p5pRT]

From @cpansprout

On Sat Jul 07 18​:35​:03 2012\, tom christiansen wrote​:

"Father Chrysostomos via RT" \perlbug\-comment@&#8203;perl\.org wrote on Sat\, 07 Jul 2012 17​:44​:46 PDT​:

I’m forwarding this to the Perl 6 language list\, so see if I can find an answer there.

I do have an answer from Damian\, which I will enclose below\, and a Rakudo result for you.

[This conversation is about how lexical subs should be implemented in Perl 5. What Perl 6 does may help in determining how to iron out the edge cases.]

[...]

This question might be more appropriate​: In this example\, which @​a does the bar subroutine see (in Perl 6)?

sub foo \{
    my @​a = \(1\,2\,3\);
    my sub bar \{ say @​a \};
    @​a := \[4\,5\,6\];
    bar\(\);
\}

The answer to your immediate question is that if you call foo()\, it prints out 456 under Rakudo.

Thank you. So the bar sub seems to be closing over the name @​a (the container/variable slot/pad entry/whatever)\, rather than the actual array itself.

Since I don’t have it installed\, could you tell me what this does?

  sub foo {   my @​a = (1\,2\,3);   my sub bar { say @​a };   bar();   @​a := [4\,5\,6];   bar();   }   foo();

And this?

  sub foo {   my @​a = (1\,2\,3);   bar();   @​a := [4\,5\,6];   bar();   my sub bar { say @​a };   }   foo();

And this?

  sub foo {   my @​a = (1\,2\,3);   my sub bar { say @​a };   my $bar = &bar;   $bar(); # is this syntax right?   @​a := [4\,5\,6];   $bar();   }   foo();

Following is Damian's answer to my question\, shared with permission.

--tom

From&#8203;:          Damian Conway \<damian@&#8203;conway\.org>
To&#8203;:            Tom Christiansen \<tchrist@&#8203;perl\.com>
CC&#8203;:            Larry Wall \<larry@&#8203;wall\.org>
Date&#8203;:          Sun\, 08 Jul 2012 07&#8203;:17&#8203;:19 \+1000
Delivery\-Date&#8203;: Sat\, 07 Jul 2012 15&#8203;:19&#8203;:09
Subject&#8203;:       Re&#8203;: my subs and state vars
In\-Reply\-To&#8203;:   \<22255\.1341691089@&#8203;chthon>

X\-Spam\-Status&#8203;: No\, score=\-102\.6 required=4\.5

tests=BAYES_00\,RCVD_IN_DNSWL_LOW\, USER_IN_WHITELIST autolearn=ham version=3.3.0

X\-Google\-Sender\-Auth​: UHLwfgo2kyvv2prdl6qJm\-RfLF8
Content\-Type​:  text/plain; charset=ISO\-8859\-1

> It looks like perl5 may be close to having my subs\, but a puzzle
> has emerged about how in some circumstances to treat state
> variables  within those\.  \[I'm pretty sure that perl6 has

thought

this through thoroughly\, but [I] am personally unfamiliar with the outcome of said contemplations.]

I bet you aren't\, though. Any ideas or clues?

The right things to do \(and what Rakudo actually does\) is to treat
lexical subs as lexically scoped \*instances\* of the specified sub
within the current surrounding block\.

That is​: a lexical sub is like a "my" var\, in that you get a new

one each time the surrounding block is executed. Rather than like an "our" variable\, where you get a new lexically scoped alias to the same package scoped variable.

By that reasoning\, state vars inside a my sub must belong to each
instance of the sub\, just as state vars inside anonymous subs

belong to each instance of the anonymous sub.

Another way of thinking about what Perl 6 does is that​:

    my sub foo \{ whatever\(\) \}

is just syntactic sugar for​:

    my &foo := sub \{ whatever\(\) \}

Does that mean I cannot call it before it is declared?

That is​: create a lexically scoped Code object and alias it at

run-time to an anonymous subroutine. So the rules for state variables inside lexical subs *must* be the same as the rules for state variables inside anonymous subs\, since they're actually just two ways of creating the same thing.

I see.

With this approach\, in Perl 6 it's easy to specify exactly what

you want​:

    sub recount\_from \($n\) \{

        my sub counter \{
            state $count = $n;   \# Each instance of &counter has

its own count say $count--; die if $count == 0; }

        while prompt "recount $n> " \{
            counter;
        \}
    \}

vs​:

    sub first\_count\_down\_from \($n\) \{

        state $count = $n;       \# All instances of &counter share

a common count

        my sub counter \{
            say $count\-\-;
            die if $count == 0;
        \}

        while prompt "first count $n> " \{
            counter;
        \}
    \}

Feel free to forward the above to anyone who might find it useful\.

What I am really trying to find out is when the subroutine is actually cloned\, and whether there can be multiple clones within a single call of the enclosing sub.

--

Father Chrysostomos

[p5pRT]

From [Unknown Contact. See original ticket]

On Sat Jul 07 18​:35​:03 2012\, tom christiansen wrote​:

"Father Chrysostomos via RT" \perlbug\-comment@&#8203;perl\.org wrote on Sat\, 07 Jul 2012 17​:44​:46 PDT​:

I’m forwarding this to the Perl 6 language list\, so see if I can find an answer there.

I do have an answer from Damian\, which I will enclose below\, and a Rakudo result for you.

[This conversation is about how lexical subs should be implemented in Perl 5. What Perl 6 does may help in determining how to iron out the edge cases.]

[...]

This question might be more appropriate​: In this example\, which @​a does the bar subroutine see (in Perl 6)?

sub foo \{
    my @​a = \(1\,2\,3\);
    my sub bar \{ say @​a \};
    @​a := \[4\,5\,6\];
    bar\(\);
\}

The answer to your immediate question is that if you call foo()\, it prints out 456 under Rakudo.

Thank you. So the bar sub seems to be closing over the name @​a (the container/variable slot/pad entry/whatever)\, rather than the actual array itself.

Since I don’t have it installed\, could you tell me what this does?

  sub foo {   my @​a = (1\,2\,3);   my sub bar { say @​a };   bar();   @​a := [4\,5\,6];   bar();   }   foo();

And this?

  sub foo {   my @​a = (1\,2\,3);   bar();   @​a := [4\,5\,6];   bar();   my sub bar { say @​a };   }   foo();

And this?

  sub foo {   my @​a = (1\,2\,3);   my sub bar { say @​a };   my $bar = &bar;   $bar(); # is this syntax right?   @​a := [4\,5\,6];   $bar();   }   foo();

Following is Damian's answer to my question\, shared with permission.

--tom

From&#8203;:          Damian Conway \<damian@&#8203;conway\.org>
To&#8203;:            Tom Christiansen \<tchrist@&#8203;perl\.com>
CC&#8203;:            Larry Wall \<larry@&#8203;wall\.org>
Date&#8203;:          Sun\, 08 Jul 2012 07&#8203;:17&#8203;:19 \+1000
Delivery\-Date&#8203;: Sat\, 07 Jul 2012 15&#8203;:19&#8203;:09
Subject&#8203;:       Re&#8203;: my subs and state vars
In\-Reply\-To&#8203;:   \<22255\.1341691089@&#8203;chthon>

X\-Spam\-Status&#8203;: No\, score=\-102\.6 required=4\.5

tests=BAYES_00\,RCVD_IN_DNSWL_LOW\, USER_IN_WHITELIST autolearn=ham version=3.3.0

X\-Google\-Sender\-Auth​: UHLwfgo2kyvv2prdl6qJm\-RfLF8
Content\-Type​:  text/plain; charset=ISO\-8859\-1

> It looks like perl5 may be close to having my subs\, but a puzzle
> has emerged about how in some circumstances to treat state
> variables  within those\.  \[I'm pretty sure that perl6 has

thought

this through thoroughly\, but [I] am personally unfamiliar with the outcome of said contemplations.]

I bet you aren't\, though. Any ideas or clues?

The right things to do \(and what Rakudo actually does\) is to treat
lexical subs as lexically scoped \*instances\* of the specified sub
within the current surrounding block\.

That is​: a lexical sub is like a "my" var\, in that you get a new

one each time the surrounding block is executed. Rather than like an "our" variable\, where you get a new lexically scoped alias to the same package scoped variable.

By that reasoning\, state vars inside a my sub must belong to each
instance of the sub\, just as state vars inside anonymous subs

belong to each instance of the anonymous sub.

Another way of thinking about what Perl 6 does is that​:

    my sub foo \{ whatever\(\) \}

is just syntactic sugar for​:

    my &foo := sub \{ whatever\(\) \}

Does that mean I cannot call it before it is declared?

That is​: create a lexically scoped Code object and alias it at

run-time to an anonymous subroutine. So the rules for state variables inside lexical subs *must* be the same as the rules for state variables inside anonymous subs\, since they're actually just two ways of creating the same thing.

I see.

With this approach\, in Perl 6 it's easy to specify exactly what

you want​:

    sub recount\_from \($n\) \{

        my sub counter \{
            state $count = $n;   \# Each instance of &counter has

its own count say $count--; die if $count == 0; }

        while prompt "recount $n> " \{
            counter;
        \}
    \}

vs​:

    sub first\_count\_down\_from \($n\) \{

        state $count = $n;       \# All instances of &counter share

a common count

        my sub counter \{
            say $count\-\-;
            die if $count == 0;
        \}

        while prompt "first count $n> " \{
            counter;
        \}
    \}

Feel free to forward the above to anyone who might find it useful\.

What I am really trying to find out is when the subroutine is actually cloned\, and whether there can be multiple clones within a single call of the enclosing sub.

--

Father Chrysostomos

[p5pRT]

From tchrist@perl.com

"Father Chrysostomos via RT" \perlbug\-comment@&#8203;perl\.org wrote   on Sat\, 07 Jul 2012 18​:54​:15 PDT​:

Thank you. So the bar sub seems to be closing over the name @​a (the container/variable slot/pad entry/whatever)\, rather than the actual array itself.

Since I don't have it installed\, could you tell me what this does?

All three of those say the same thing​:

  123   456

--tom

[p5pRT]

From damian@conway.org

Father Chrysostomos asked​:

What I am really trying to find out is when the subroutine is actually cloned\,

Yes. It is supposed to be (or at least must *appear* to be)\, and currently is (or appears to be) in Rakudo.

and whether there can be multiple clones within a single call of the enclosing sub.

Yes. For example\, a lexical sub might be declared in a loop inside the enclosing sub\, in which case it should produce multiple instances\, one per iteration.

For example\, this​:

  sub outer_sub () {   for (1..3) {   state $call_num = 1;   my sub inner_sub {   state $inner_state = (1..100).pick; # i.e. random number   say " [call {$call_num++}] \$inner_state = $inner_state";   }

  say "\nsub id​: "\, &inner_sub.id;   inner_sub();   inner_sub();   }   }

  outer_sub();

produces​:

  sub id​: -4628941774842748435   [call 1] $inner_state = 89   [call 2] $inner_state = 89

  sub id​: -4628941774848253711   [call 3] $inner_state = 16   [call 4] $inner_state = 16

  sub id​: -4628941774839825925   [call 5] $inner_state = 26   [call 6] $inner_state = 26

under Rakudo

BTW\, Both the above "yes" answers are consistent with (and can be inferred from) the previous explanation that​:

  my sub foo { whatever() }

is just a syntactic convenience for​:

  my &foo := sub { whatever() }

HTH\,

Damian

[p5pRT]

From @cpansprout

On Sat Jul 07 22​:23​:16 2012\, thoughtstream wrote​:

Father Chrysostomos asked​:

What I am really trying to find out is when the subroutine is actually cloned\,

Yes. It is supposed to be (or at least must *appear* to be)\, and currently is (or appears to be) in Rakudo.

I said when\, not whether. :-)

and whether there can be multiple clones within a single call of the enclosing sub.

Yes. For example\, a lexical sub might be declared in a loop inside the enclosing sub\, in which case it should produce multiple instances\, one per iteration.

For example\, this​:

sub outer\_sub \(\) \{
    for \(1\.\.3\) \{
        state $call\_num = 1;
        my sub inner\_sub \{
            state $inner\_state = \(1\.\.100\)\.pick; \# i\.e\. random number
            say "    \[call \{$call\_num\+\+\}\] \\$inner\_state =

$inner_state"; }

        say "\\nsub id​: "\, &inner\_sub\.id;
        inner\_sub\(\);
        inner\_sub\(\);
    \}
\}

outer\_sub\(\);

produces​:

sub id​: \-4628941774842748435
    \[call 1\] $inner\_state = 89
    \[call 2\] $inner\_state = 89

sub id​: \-4628941774848253711
    \[call 3\] $inner\_state = 16
    \[call 4\] $inner\_state = 16

sub id​: \-4628941774839825925
    \[call 5\] $inner\_state = 26
    \[call 6\] $inner\_state = 26

under Rakudo

Thank you.

Does Perl 6 have an equivalent to this?

  my $x;   for $x(1..10) {}

In this case\, the loop reuses the $x slot in the pad\, making the existing $x name an alias to a different scalar. But\, due to the way Perl 5 closures work\, it is\, for all tents and porpoises\, the same as this​:

  my $x;   for my $x(1..10) {}

BTW\, Both the above "yes" answers are consistent with (and can be inferred from) the previous explanation that​:

    my sub foo \{ whatever\(\) \}

is just a syntactic convenience for​:

    my &foo := sub \{ whatever\(\) \}

Except that my sub foo happens upon block entry\, right?

HTH\,

It does\, but I am still trying to wrap my head around the fundamental difference between 5 and 6 with regard to closures.

In Perl 5\, $] in a piece of code is bound to *]\, not $]\, so it sees changes made by local($]) (which actually puts a completely new scalar in the *]{SCALAR} slot). But ‘my $x; sub { $x }’ is bound\, not to the $x slot in the outer block/sub/file\, but to the actual scalar itself.

It seems that Perl 6 closures close over the slot\, not the scalar/array/etc. Is that right?

Anyway\, I think I know how to implement this now.

The first time a ‘my’ sub is referenced or called\, it is cloned. The clone is stored for reuse\, but that storage is localised to the current block.

--

Father Chrysostomos

[p5pRT]

From [Unknown Contact. See original ticket]

On Sat Jul 07 22​:23​:16 2012\, thoughtstream wrote​:

Father Chrysostomos asked​:

What I am really trying to find out is when the subroutine is actually cloned\,

Yes. It is supposed to be (or at least must *appear* to be)\, and currently is (or appears to be) in Rakudo.

I said when\, not whether. :-)

and whether there can be multiple clones within a single call of the enclosing sub.

Yes. For example\, a lexical sub might be declared in a loop inside the enclosing sub\, in which case it should produce multiple instances\, one per iteration.

For example\, this​:

sub outer\_sub \(\) \{
    for \(1\.\.3\) \{
        state $call\_num = 1;
        my sub inner\_sub \{
            state $inner\_state = \(1\.\.100\)\.pick; \# i\.e\. random number
            say "    \[call \{$call\_num\+\+\}\] \\$inner\_state =

$inner_state"; }

        say "\\nsub id​: "\, &inner\_sub\.id;
        inner\_sub\(\);
        inner\_sub\(\);
    \}
\}

outer\_sub\(\);

produces​:

sub id​: \-4628941774842748435
    \[call 1\] $inner\_state = 89
    \[call 2\] $inner\_state = 89

sub id​: \-4628941774848253711
    \[call 3\] $inner\_state = 16
    \[call 4\] $inner\_state = 16

sub id​: \-4628941774839825925
    \[call 5\] $inner\_state = 26
    \[call 6\] $inner\_state = 26

under Rakudo

Thank you.

Does Perl 6 have an equivalent to this?

  my $x;   for $x(1..10) {}

In this case\, the loop reuses the $x slot in the pad\, making the existing $x name an alias to a different scalar. But\, due to the way Perl 5 closures work\, it is\, for all tents and porpoises\, the same as this​:

  my $x;   for my $x(1..10) {}

BTW\, Both the above "yes" answers are consistent with (and can be inferred from) the previous explanation that​:

    my sub foo \{ whatever\(\) \}

is just a syntactic convenience for​:

    my &foo := sub \{ whatever\(\) \}

Except that my sub foo happens upon block entry\, right?

HTH\,

It does\, but I am still trying to wrap my head around the fundamental difference between 5 and 6 with regard to closures.

In Perl 5\, $] in a piece of code is bound to *]\, not $]\, so it sees changes made by local($]) (which actually puts a completely new scalar in the *]{SCALAR} slot). But ‘my $x; sub { $x }’ is bound\, not to the $x slot in the outer block/sub/file\, but to the actual scalar itself.

It seems that Perl 6 closures close over the slot\, not the scalar/array/etc. Is that right?

Anyway\, I think I know how to implement this now.

The first time a ‘my’ sub is referenced or called\, it is cloned. The clone is stored for reuse\, but that storage is localised to the current block.

--

Father Chrysostomos

[p5pRT]

From damian@conway.org

Father Chrysostomos pointed out​:

I said when\, not whether. :-)

Isn't that just typical of me​: confusing ontology with chronology. ;-)

I'm afraid don't know the implementation details for Rakudo. It may be bound as the surrounding block is entered\, or perhaps just-in-time when the Code object is first accessed in some way.

Does Perl 6 have an equivalent to this?

my $x;
for $x\(1\.\.10\) \{\}

In Perl 6 that's​:

  my $x;   for 1..10 -> $x {}

And\, as in Perl 5\, they're two separate variables.

The iterator $x is a parameter of the loop block and acts as a readonly alias to each successive iterated value\, unless you write​:

  for 1..10 \<-> $x {}

in which case it's a read-write alias to each iterated value.

Loop iterators are never package-scoped in Perl 6.

is just a syntactic convenience for​:

    my &foo := sub \{ whatever\(\) \}

Except that my sub foo happens upon block entry\, right?

Quite so. So the equivalence is more precisely​:

  ENTER my &foo := sub { whatever() };

except that the named code object is visible throughout its surrounding block (even before its declaration point).

So I guess that equivalence I keep referring to is a little shakey. ;-)

In Perl 5\, $] in a piece of code is bound to *]\, not $]\, so it sees changes made by local($]) (which actually puts a completely new scalar in the *]{SCALAR} slot). But ‘my $x; sub { $x }’ is bound\, not to the $x slot in the outer block/sub/file\, but to the actual scalar itself.

It seems that Perl 6 closures close over the slot\, not the scalar/array/etc. Is that right?

That's right. This​:

  my $x = 1;   my $x_prime = -1;

  sub foo { say $x }

  foo();

  $x := $x_prime;

  foo();

prints​:

  1   -1

...indicating that the $x in foo is bound to the slot\, not to any var implementation currently in the slot. The way I would have described it is that binding is by name\, not by reference.

Damian

[p5pRT]

From @moritz

On 07/08/2012 11​:12 AM\, Damian Conway wrote​:

Father Chrysostomos pointed out​:

I said when\, not whether. :-)

Isn't that just typical of me​: confusing ontology with chronology. ;-)

I'm afraid don't know the implementation details for Rakudo. It may be bound as the surrounding block is entered\, or perhaps just-in-time when the Code object is first accessed in some way.

When the surrounding block is entered.

http​://perlcabal.org/syn/S04.html#When_is_a_closure_not_a_closure

"All remaining blocks are conceptually cloned into closures as soon as the lexical scope containing them is entered."

Cheers\, Moritz

[p5pRT]

From @cpansprout

On Sun Jul 08 02​:13​:13 2012\, thoughtstream wrote​:

Father Chrysostomos pointed out​:

I said when\, not whether. :-)

Isn't that just typical of me​: confusing ontology with chronology. ;-)

I'm afraid don't know the implementation details for Rakudo. It may be bound as the surrounding block is entered\, or perhaps just-in-time when the Code object is first accessed in some way.

Does Perl 6 have an equivalent to this?

my $x;
for $x\(1\.\.10\) \{\}

In Perl 6 that's​:

my $x;
for 1\.\.10 \-> $x \{\}

And\, as in Perl 5\, they're two separate variables.

But by using the term ‘variable’\, which is ambiguous\, you are not answering my question! :-)

In Perl 5\, ‘my $x; for $x(1..10){}’ is only equivalent to ‘my $x; for my $x(1..10){}’ because we don’t have the := operator\, so there is never any possibility to see the difference\, even though things happen differently underneath.

Does

  my $x;   for 1..10 -> $x {}

cause the existing name $x to refer temporarily to each of the numbers\, or is a new $x name created?

What does this do?

  my $x;   my sub f { say $x }   for 1..10 -> $x { f(); }

--

Father Chrysostomos

[p5pRT]

From [Unknown Contact. See original ticket]

On Sun Jul 08 02​:13​:13 2012\, thoughtstream wrote​:

Father Chrysostomos pointed out​:

I said when\, not whether. :-)

Isn't that just typical of me​: confusing ontology with chronology. ;-)

I'm afraid don't know the implementation details for Rakudo. It may be bound as the surrounding block is entered\, or perhaps just-in-time when the Code object is first accessed in some way.

Does Perl 6 have an equivalent to this?

my $x;
for $x\(1\.\.10\) \{\}

In Perl 6 that's​:

my $x;
for 1\.\.10 \-> $x \{\}

And\, as in Perl 5\, they're two separate variables.

But by using the term ‘variable’\, which is ambiguous\, you are not answering my question! :-)

In Perl 5\, ‘my $x; for $x(1..10){}’ is only equivalent to ‘my $x; for my $x(1..10){}’ because we don’t have the := operator\, so there is never any possibility to see the difference\, even though things happen differently underneath.

Does

  my $x;   for 1..10 -> $x {}

cause the existing name $x to refer temporarily to each of the numbers\, or is a new $x name created?

What does this do?

  my $x;   my sub f { say $x }   for 1..10 -> $x { f(); }

--

Father Chrysostomos

[p5pRT]

From @moritz

On 07/08/2012 09​:57 PM\, Father Chrysostomos via RT wrote​:

my $x;
my sub f \{ say $x \}
for 1\.\.10 \-> $x \{ f\(\); \}

It prints

Any() Any() Any() Any() Any() Any() Any() Any() Any() Any()

(because Any is the default value in uninitialized variables).

As an aside\, you can run short Perl 6 scripts on IRC (on irc.perl.org and irc.freenode.org) with something like

/msg p6eval p6​: my $x; sub sub f { say $x }; for 1..10 -> $x { f() }

This runs it through both rakudo and niecza.

If you want\, I can also send the bot into #p5p.

Cheers\, Moritz

[p5pRT]

From damian@conway.org

But by using the term ‘variable’\, which is ambiguous\, you are not answering my question! :-)

Sorry. I tend to think of *every* variable name as merely being an alias for some underlying storage mechanism. ;-)

Does

my $x;
for 1\.\.10 \-> $x \{\}

cause the existing name $x to refer temporarily to each of the numbers\, or is a new $x name created?

A new one is created (each time through the loop).

What does this do?

my $x;
my sub f \{ say $x \}
for 1\.\.10 \-> $x \{ f\(\); \}

It prints 'Any()' ten times (i.e. the equivalent of printing ten Perl 5 undefs).

The two $x's definitely exist at the same time during the loop. For example\, this​:

  my $x = 'outer x';   my sub f { say $x }

  for 1..10 -> $x {   print $x\, "​: ";   f();   }

prints​:

  1​: outer x   2​: outer x   3​: outer x   4​: outer x   5​: outer x   6​: outer x   7​: outer x   8​: outer x   9​: outer x   10​: outer x

Damian

[p5pRT]

From @cpansprout

Based on feedback from Damian Conway about how these work in Perl 6 and how they interact with state variables\, I am now of the opinion that ‘my’ subs need to close over their variables whenever the enclosing block is entered.

Since ‘my’ creates a new variable for each iteration of a loop\, ‘my sub’ should create a new sub for each iteration.

And for consistency with my variables\, a lexical sub that is closed over by a named sub needs to appear ‘pre-cloned’\, unless it is inside an anonymous sub.

In Perl 6\, the sub is created when the enclosing block is entered\, which is when the sub becomes visible. Perl 5 scoping is different. You don’t see the sub until after the ‘my sub’. So I was going to say that the variables are closed over when the name comes into scope. But that model falls apart when you use goto to jump over the sub definition\, either forwards or backwards. If you do that\, when exactly does the name become visible?

My earlier idea for dwimming with​:

  my $x;   my sub f { warn $x };   for $x (1..10) { f(); }

and making it print 1 to 10 cannot work\, because it would require cloning the sub every time it is called. That cannot happen\, because it may contain state variables\, which would not be shared between instances. (for (1..10) { my sub f { state $x } } *should* create separate state variables\, otoh\, because there are ten separate subs.)

So this is what we end up with (these are all the weirdest edge cases I could think of)​:

Assume this for all examples​:

  sub p { print shift//undef\, "\n" }

The f sub binds to the original $x\, printing ‘undef’ ten times​:

  my $x;   my sub f { p $x }   for $x(1..10) {   f();   }

Here the f sub binds to the new $x in each iteration\, printing 1 to 10​:

  my $x;   for $x(1..10) {   my sub f { p $x }   f();   }

The named sub closes over the first instance of m\, as it would with my $scalar\, causing "yes" to be printed​:

  {   my sub m { p "yes" }   sub o { m() }   }   o();

Here\, the block containing the m sub has never been entered\, but its enclosing CV is running\, so the m sub is visible\, but closes over an undefined $x\, printing ‘undef’​:

  o();   {   my $x = 42;   my sub m { p $x }   sub o { m() }   }

In this example\, the block containing the m sub is the body of an anonymous sub\, so we get a ‘Subroutine m is not available’ warning\, and ‘Undefined subroutine &m called’​:

  o();   sub {   my sub m { p "not ok" }   sub o { m() }   }

Since this prints 3​:

  o();   BEGIN {   my $x = 3;   sub o { p $x }   }

This should do the same​:

  o();   BEGIN {   my $x = 3;   my sub m { p $x }   sub o { m() }   }

To go back to the earlier example (without the BEGIN block or anon sub)\, if we move the call after the block\, we get 3 here\, too​:

  {   my $x = 3;   my sub m { p $x }   sub o { m() }   }   o();

just as we do in this case​:

  {   my $x = 3;   sub o { p $x }   }   o();

--

Father Chrysostomos

[p5pRT]

From [Unknown Contact. See original ticket]

Based on feedback from Damian Conway about how these work in Perl 6 and how they interact with state variables\, I am now of the opinion that ‘my’ subs need to close over their variables whenever the enclosing block is entered.

Since ‘my’ creates a new variable for each iteration of a loop\, ‘my sub’ should create a new sub for each iteration.

And for consistency with my variables\, a lexical sub that is closed over by a named sub needs to appear ‘pre-cloned’\, unless it is inside an anonymous sub.

In Perl 6\, the sub is created when the enclosing block is entered\, which is when the sub becomes visible. Perl 5 scoping is different. You don’t see the sub until after the ‘my sub’. So I was going to say that the variables are closed over when the name comes into scope. But that model falls apart when you use goto to jump over the sub definition\, either forwards or backwards. If you do that\, when exactly does the name become visible?

My earlier idea for dwimming with​:

  my $x;   my sub f { warn $x };   for $x (1..10) { f(); }

and making it print 1 to 10 cannot work\, because it would require cloning the sub every time it is called. That cannot happen\, because it may contain state variables\, which would not be shared between instances. (for (1..10) { my sub f { state $x } } *should* create separate state variables\, otoh\, because there are ten separate subs.)

So this is what we end up with (these are all the weirdest edge cases I could think of)​:

Assume this for all examples​:

  sub p { print shift//undef\, "\n" }

The f sub binds to the original $x\, printing ‘undef’ ten times​:

  my $x;   my sub f { p $x }   for $x(1..10) {   f();   }

Here the f sub binds to the new $x in each iteration\, printing 1 to 10​:

  my $x;   for $x(1..10) {   my sub f { p $x }   f();   }

The named sub closes over the first instance of m\, as it would with my $scalar\, causing "yes" to be printed​:

  {   my sub m { p "yes" }   sub o { m() }   }   o();

Here\, the block containing the m sub has never been entered\, but its enclosing CV is running\, so the m sub is visible\, but closes over an undefined $x\, printing ‘undef’​:

  o();   {   my $x = 42;   my sub m { p $x }   sub o { m() }   }

In this example\, the block containing the m sub is the body of an anonymous sub\, so we get a ‘Subroutine m is not available’ warning\, and ‘Undefined subroutine &m called’​:

  o();   sub {   my sub m { p "not ok" }   sub o { m() }   }

Since this prints 3​:

  o();   BEGIN {   my $x = 3;   sub o { p $x }   }

This should do the same​:

  o();   BEGIN {   my $x = 3;   my sub m { p $x }   sub o { m() }   }

To go back to the earlier example (without the BEGIN block or anon sub)\, if we move the call after the block\, we get 3 here\, too​:

  {   my $x = 3;   my sub m { p $x }   sub o { m() }   }   o();

just as we do in this case​:

  {   my $x = 3;   sub o { p $x }   }   o();

--

Father Chrysostomos

[p5pRT]

From @iabyn

On Sun\, Jul 08\, 2012 at 01​:38​:21PM -0700\, Father Chrysostomos via RT wrote​:

Based on feedback from Damian Conway about how these work in Perl 6 and how they interact with state variables\, I am now of the opinion that ‘my’ subs need to close over their variables whenever the enclosing block is entered. [snip] Here\, the block containing the m sub has never been entered\, but its enclosing CV is running\, so the m sub is visible\, but closes over an undefined $x\, printing ‘undef’​:

o\(\);
\{
    my $x = 42;
    my sub m \{ p $x \}
    sub o \{ m\(\) \}
\}

I would expect it to instead to complain about calling an undefined function instead; i.e. at the point of calling m()\, m exists but has not yet been defined.

Other than that\, I agree with all your examples. Note that if you change the one example above to my interpretation\, then *every* edge case you listed behaves identically as if you'd replaced

  my sub m {...}

with

  my $m = sub { ...};

Finally\, what do you intend to happen regarding cloning with code like​:

  FOO​:   my sub m { ... }   m();   goto FOO if ...;

I think redo may invoke a similar issue.

-- Technology is dominated by two types of people​: those who understand what they do not manage\, and those who manage what they do not understand.

[p5pRT]

From @cpansprout

On Fri Jul 13 13​:21​:05 2012\, davem wrote​:

On Sun\, Jul 08\, 2012 at 01​:38​:21PM -0700\, Father Chrysostomos via RT wrote​:

Based on feedback from Damian Conway about how these work in Perl 6 and how they interact with state variables\, I am now of the opinion that ‘my’ subs need to close over their variables whenever the enclosing block is entered. [snip] Here\, the block containing the m sub has never been entered\, but its enclosing CV is running\, so the m sub is visible\, but closes over an undefined $x\, printing ‘undef’​:

o\(\);
\{
    my $x = 42;
    my sub m \{ p $x \}
    sub o \{ m\(\) \}
\}

I would expect it to instead to complain about calling an undefined function instead; i.e. at the point of calling m()\, m exists but has not yet been defined.

Other than that\, I agree with all your examples. Note that if you change the one example above to my interpretation\, then *every* edge case you listed behaves identically as if you'd replaced

my sub m \{\.\.\.\}

with

my $m = sub \{ \.\.\.\};

Finally\, what do you intend to happen regarding cloning with code like​:

FOO​: my sub m { ... } m(); goto FOO if ...;

I think redo may invoke a similar issue.

Like ‘sub foo’\, ‘my sub foo’ will be a compile-time declaration. For scoping to obey its usual rules\, this will have to work​:

  my sub m;   sub m { ... } # attaches to lexical &m

And for ‘sub m’ to be a compile-time directive only sometimes does not seem right.

So I was going to clone my subs on block entry (already partially done locally). In the example you gave\, the ‘my sub’ line is never actually executed. So you get the same m sub after the goto.

Likewise\, this should work\, too​:

  my sub m;   m();   sub m { ... }

What happens in cases like this?

  sub foo {   my $x;   sub bar { ... }   }

Does bar see the first $x?

Now back to this example​:

o\(\);
\{
    my $x = 42;
    my sub m \{ p $x \}
    sub o \{ m\(\) \}
\}

What if you change ‘my sub’ to ‘state sub’? What would you expect then?

Also\, if the o sub can see the first instance of $x\, should it not see the first &m? (Don’t forget that you can’t exactly assign over a sub.)

And another issue which is currently impeding progress​:

Following existing scoping rules\, but with a twist due to ‘sub m’ being a compile-time declaration\, I would expect

  my sub m;   my sub n {   sub m { ... }   }

to define the m sub at compile time. What should happen in that case is fairly straightforward. The declaration works its way out to the pad entry\, just like regular variable lookup.

But then things become very confusing in cases like these​:

  sub {   my sub m;   sub {   eval 'sub m {...}';   }   }->()();

  sub {   my sub m;   sub foo {   eval 'sub m {...}';   }   }   foo();

How far does the declaration make its way outward looking for its pad entry? Does it stop at the first running sub?

I’m tempted to ignore that case for now and just see how the implementation ends up doing it.

This is all making my head spin.

--

Father Chrysostomos

[p5pRT]

From [Unknown Contact. See original ticket]

On Fri Jul 13 13​:21​:05 2012\, davem wrote​:

On Sun\, Jul 08\, 2012 at 01​:38​:21PM -0700\, Father Chrysostomos via RT wrote​:

Based on feedback from Damian Conway about how these work in Perl 6 and how they interact with state variables\, I am now of the opinion that ‘my’ subs need to close over their variables whenever the enclosing block is entered. [snip] Here\, the block containing the m sub has never been entered\, but its enclosing CV is running\, so the m sub is visible\, but closes over an undefined $x\, printing ‘undef’​:

o\(\);
\{
    my $x = 42;
    my sub m \{ p $x \}
    sub o \{ m\(\) \}
\}

I would expect it to instead to complain about calling an undefined function instead; i.e. at the point of calling m()\, m exists but has not yet been defined.

Other than that\, I agree with all your examples. Note that if you change the one example above to my interpretation\, then *every* edge case you listed behaves identically as if you'd replaced

my sub m \{\.\.\.\}

with

my $m = sub \{ \.\.\.\};

Finally\, what do you intend to happen regarding cloning with code like​:

FOO​: my sub m { ... } m(); goto FOO if ...;

I think redo may invoke a similar issue.

Like ‘sub foo’\, ‘my sub foo’ will be a compile-time declaration. For scoping to obey its usual rules\, this will have to work​:

  my sub m;   sub m { ... } # attaches to lexical &m

And for ‘sub m’ to be a compile-time directive only sometimes does not seem right.

So I was going to clone my subs on block entry (already partially done locally). In the example you gave\, the ‘my sub’ line is never actually executed. So you get the same m sub after the goto.

Likewise\, this should work\, too​:

  my sub m;   m();   sub m { ... }

What happens in cases like this?

  sub foo {   my $x;   sub bar { ... }   }

Does bar see the first $x?

Now back to this example​:

o\(\);
\{
    my $x = 42;
    my sub m \{ p $x \}
    sub o \{ m\(\) \}
\}

What if you change ‘my sub’ to ‘state sub’? What would you expect then?

Also\, if the o sub can see the first instance of $x\, should it not see the first &m? (Don’t forget that you can’t exactly assign over a sub.)

And another issue which is currently impeding progress​:

Following existing scoping rules\, but with a twist due to ‘sub m’ being a compile-time declaration\, I would expect

  my sub m;   my sub n {   sub m { ... }   }

to define the m sub at compile time. What should happen in that case is fairly straightforward. The declaration works its way out to the pad entry\, just like regular variable lookup.

But then things become very confusing in cases like these​:

  sub {   my sub m;   sub {   eval 'sub m {...}';   }   }->()();

  sub {   my sub m;   sub foo {   eval 'sub m {...}';   }   }   foo();

How far does the declaration make its way outward looking for its pad entry? Does it stop at the first running sub?

I’m tempted to ignore that case for now and just see how the implementation ends up doing it.

This is all making my head spin.

--

Father Chrysostomos

[p5pRT]

From @iabyn

On Fri\, Jul 13\, 2012 at 03​:54​:44PM -0700\, Father Chrysostomos via RT wrote​: [stuff]

Before I try to reply to that\, just as a data point\, to what extent (if any) do you consider the behaviour of 'my/state sub' in the following​:

  my sub foo;   my sub foo { ... };   foo();

  state sub bar { ... };   bar();

to be different in behaviour or semantics to the following​:

  my $foo;   $foo = sub { ... };   $foo->();

  state $bar = sub { ... };   $bar->();

apart from that at run-time\, the assignment happens at the point of block entry rather than at the point where the declaration is reached?

-- The warp engines start playing up a bit\, but seem to sort themselves out after a while without any intervention from boy genius Wesley Crusher.   -- Things That Never Happen in "Star Trek" #17

[p5pRT]

From @rurban

On Wed\, Jul 18\, 2012 at 11​:30 AM\, Dave Mitchell \davem@&#8203;iabyn\.com wrote​:

On Fri\, Jul 13\, 2012 at 03​:54​:44PM -0700\, Father Chrysostomos via RT wrote​: [stuff]

Before I try to reply to that\, just as a data point\, to what extent (if any) do you consider the behaviour of 'my/state sub' in the following​:

my sub foo;
my sub foo \{ \.\.\. \};
foo\(\);

state sub bar \{ \.\.\. \};
bar\(\);

to be different in behaviour or semantics to the following​:

my $foo;
$foo = sub \{ \.\.\. \};
$foo\->\(\);

state $bar = sub \{ \.\.\. \};
$bar\->\(\);

apart from that at run-time\, the assignment happens at the point of block entry rather than at the point where the declaration is reached?

The only real usecase for lexical subs I see is for private methods in package blocks.

package bla {   my sub _get { ... }   sub get { shift->_get(); ... }   sub new { bless {@​_} } }

And $blaobj->_get is run-time invalid (unless _get is a public method upwards). Calling outside the block bla​::_get() is even a compile-time error.

decl vs assign

decl​:   my sub foo; #forward decl   my sub foo { ... };   foo();

  state sub bar { ... }; #normal decl   bar();

vs. assignment​:   my $foo; #scalar decl   $foo = sub { ... }; #oh it's a lexical cv   $foo->();

  state $bar = sub { ... };   $bar->();

to be consistent with other scalar assignments\, the 2nd case is done naturally - at the point where the declaration is reached\, and the 1st case also. Otherwise we would not need to do forward declarations.

As we are here at basic OO​: const %bla​:: (readonly stash) will also protect you from using wrong accessors\, as they will be detected at compile-time. This will hopefully convince people not to use slow and big accessor methods for everything which can be done by using simple hash accessors directly. Unless you are planning to use the MOP.

const package bla {   my $attr1; # private attribute   our $attr2; # public attribute   sub new { bless {@​_} } } $bla = new bla; $bla->{attr2} = 1; $bla->{attr1} = 0; => Attempt to access disallowed key 'attr1' in restricted hash

For your unpatched perls​: $ perl -MConst​::Fast -e'const %a => (0=>1\,1=>2); $a{2}=0'; Attempt to access disallowed key '2' in a restricted hash at -e line 1.

Because the semantics do not work yet and the syntax is awful​: $ perl -e'package a; our $a; package main; Internals​::SvREADONLY(%a​::); print $a{b}'; => no error -- Reini Urban http​://cpanel.net/ http​://www.perl-compiler.org/

[p5pRT]

From @cpansprout

On Wed Jul 18 09​:31​:10 2012\, davem wrote​:

On Fri\, Jul 13\, 2012 at 03​:54​:44PM -0700\, Father Chrysostomos via RT wrote​: [stuff]

Before I try to reply to that\, just as a data point\, to what extent (if any) do you consider the behaviour of 'my/state sub' in the following​:

my sub foo;
my sub foo \{ \.\.\. \};
foo\(\);

state sub bar \{ \.\.\. \};
bar\(\);

to be different in behaviour or semantics to the following​:

my $foo;
$foo = sub \{ \.\.\. \};
$foo\->\(\);

state $bar = sub \{ \.\.\. \};
$bar\->\(\);

apart from that at run-time\, the assignment happens at the point of block entry rather than at the point where the declaration is reached?

I hadn’t thought about it that way\, but they are more or less equivalent (except for when the assignment happens).

I also think now that a &foo pad entry visible to a named sub before that block is entered should contain a sub stub. When the block is entered\, the prototype is cloned into the existing stub\, resulting in scoping nearly identical to lexical scalars.

--

Father Chrysostomos

[p5pRT]

From [Unknown Contact. See original ticket]

On Wed Jul 18 09​:31​:10 2012\, davem wrote​:

On Fri\, Jul 13\, 2012 at 03​:54​:44PM -0700\, Father Chrysostomos via RT wrote​: [stuff]

Before I try to reply to that\, just as a data point\, to what extent (if any) do you consider the behaviour of 'my/state sub' in the following​:

my sub foo;
my sub foo \{ \.\.\. \};
foo\(\);

state sub bar \{ \.\.\. \};
bar\(\);

to be different in behaviour or semantics to the following​:

my $foo;
$foo = sub \{ \.\.\. \};
$foo\->\(\);

state $bar = sub \{ \.\.\. \};
$bar\->\(\);

apart from that at run-time\, the assignment happens at the point of block entry rather than at the point where the declaration is reached?

I hadn’t thought about it that way\, but they are more or less equivalent (except for when the assignment happens).

I also think now that a &foo pad entry visible to a named sub before that block is entered should contain a sub stub. When the block is entered\, the prototype is cloned into the existing stub\, resulting in scoping nearly identical to lexical scalars.

--

Father Chrysostomos

[p5pRT]

From @cpansprout

On Wed Jul 18 12​:31​:38 2012\, rurban wrote​:

The only real usecase for lexical subs I see is for private methods in package blocks.

package bla { my sub _get { ... } sub get { shift->_get(); ... } sub new { bless {@​_} } }

And $blaobj->_get is run-time invalid (unless _get is a public method upwards). Calling outside the block bla​::_get() is even a compile-time error.

That would be _get($blaobj)\, not $blaobj->_get. If it’s private\, it’s not really a method\, so it doesn’t add anything except confusion to call it as one. And to make ->foo starting binding over lexical subs would prevent calling a ->foo method on an unrelated object.

decl vs assign

decl​: my sub foo; #forward decl my sub foo { ... }; foo();

 state sub bar \{ \.\.\. \}; \#normal decl
 bar\(\);

vs. assignment​: my $foo; #scalar decl $foo = sub { ... }; #oh it's a lexical cv $foo->();

 state $bar = sub \{ \.\.\. \};
 $bar\->\(\);

to be consistent with other scalar assignments\, the 2nd case is done naturally - at the point where the declaration is reached\, and the 1st case also. Otherwise we would not need to do forward declarations.

Cloning the sub on block entry but making it visible only within the scope starting with the first declaration effectively does that\, but avoids the problem of ‘goto’ jumping over the declaration (either forwards or backwards).

As we are here at basic OO​: const %bla​:: (readonly stash) will also protect you from using wrong accessors\, as they will be detected at compile-time. This will hopefully convince people not to use slow and big accessor methods for everything which can be done by using simple hash accessors directly. Unless you are planning to use the MOP.

const package bla { my $attr1; # private attribute our $attr2; # public attribute sub new { bless {@​_} } } $bla = new bla; $bla->{attr2} = 1; $bla->{attr1} = 0; => Attempt to access disallowed key 'attr1' in restricted hash

You seem to be conflating variables and hash keys\, which in Perl (unlike JavaScript) are orthogonal concepts.

For your unpatched perls​: $ perl -MConst​::Fast -e'const %a => (0=>1\,1=>2); $a{2}=0'; Attempt to access disallowed key '2' in a restricted hash at -e line 1.

Because the semantics do not work yet and the syntax is awful​: $ perl -e'package a; our $a; package main; Internals​::SvREADONLY(%a​::); print $a{b}'; => no error

SvREADONLY means about three different things (restricted hash\, COW\, and actually read-only). If we want that to work\, we might need to disentangle those meanings first.

--

Father Chrysostomos

[p5pRT]

From [Unknown Contact. See original ticket]

On Wed Jul 18 12​:31​:38 2012\, rurban wrote​:

The only real usecase for lexical subs I see is for private methods in package blocks.

package bla { my sub _get { ... } sub get { shift->_get(); ... } sub new { bless {@​_} } }

And $blaobj->_get is run-time invalid (unless _get is a public method upwards). Calling outside the block bla​::_get() is even a compile-time error.

That would be _get($blaobj)\, not $blaobj->_get. If it’s private\, it’s not really a method\, so it doesn’t add anything except confusion to call it as one. And to make ->foo starting binding over lexical subs would prevent calling a ->foo method on an unrelated object.

decl vs assign

decl​: my sub foo; #forward decl my sub foo { ... }; foo();

 state sub bar \{ \.\.\. \}; \#normal decl
 bar\(\);

vs. assignment​: my $foo; #scalar decl $foo = sub { ... }; #oh it's a lexical cv $foo->();

 state $bar = sub \{ \.\.\. \};
 $bar\->\(\);

to be consistent with other scalar assignments\, the 2nd case is done naturally - at the point where the declaration is reached\, and the 1st case also. Otherwise we would not need to do forward declarations.

Cloning the sub on block entry but making it visible only within the scope starting with the first declaration effectively does that\, but avoids the problem of ‘goto’ jumping over the declaration (either forwards or backwards).

As we are here at basic OO​: const %bla​:: (readonly stash) will also protect you from using wrong accessors\, as they will be detected at compile-time. This will hopefully convince people not to use slow and big accessor methods for everything which can be done by using simple hash accessors directly. Unless you are planning to use the MOP.

const package bla { my $attr1; # private attribute our $attr2; # public attribute sub new { bless {@​_} } } $bla = new bla; $bla->{attr2} = 1; $bla->{attr1} = 0; => Attempt to access disallowed key 'attr1' in restricted hash

You seem to be conflating variables and hash keys\, which in Perl (unlike JavaScript) are orthogonal concepts.

For your unpatched perls​: $ perl -MConst​::Fast -e'const %a => (0=>1\,1=>2); $a{2}=0'; Attempt to access disallowed key '2' in a restricted hash at -e line 1.

Because the semantics do not work yet and the syntax is awful​: $ perl -e'package a; our $a; package main; Internals​::SvREADONLY(%a​::); print $a{b}'; => no error

SvREADONLY means about three different things (restricted hash\, COW\, and actually read-only). If we want that to work\, we might need to disentangle those meanings first.

--

Father Chrysostomos

[p5pRT]

From @cpansprout

On Wed Jul 25 16​:00​:54 2012\, sprout wrote​:

I also think now that a &foo pad entry visible to a named sub before that block is entered should contain a sub stub. When the block is entered\, the prototype is cloned into the existing stub\, resulting in scoping nearly identical to lexical scalars.

But here is an interesting question. What happens in this case?

my sub foo ($) {} BEGIN {   warn defined &foo ? "defined" : "undefined";

  # What happens here???   warn prototype \&foo; }

From an implementation standpoint\, it is easier for the prototype not to be visible before the scope is entered\, making it comparable to​:

sub foo; # at compile time eval 'sub foo($) {}' # on scope entry

rather than​:

sub foo($); eval 'sub foo($) {}'

--

Father Chrysostomos

[p5pRT]

From [Unknown Contact. See original ticket]

On Wed Jul 25 16​:00​:54 2012\, sprout wrote​:

I also think now that a &foo pad entry visible to a named sub before that block is entered should contain a sub stub. When the block is entered\, the prototype is cloned into the existing stub\, resulting in scoping nearly identical to lexical scalars.

But here is an interesting question. What happens in this case?

my sub foo ($) {} BEGIN {   warn defined &foo ? "defined" : "undefined";

  # What happens here???   warn prototype \&foo; }

From an implementation standpoint\, it is easier for the prototype not to be visible before the scope is entered\, making it comparable to​:

sub foo; # at compile time eval 'sub foo($) {}' # on scope entry

rather than​:

sub foo($); eval 'sub foo($) {}'

--

Father Chrysostomos