Request to deprecate untwigiled attributes

[librasteve]

Problem:

The untwigiled variant of has attributes unbalances the symmetry of $. public and $! private attributes.

---

This issue is the new home of the one I posted wrongly over here https://github.com/rakudo/rakudo/issues/5139 (now closed)

---

Details:

Coming in from this reddit post and raising a new Problem Solving issue here as suggested.

I am reflecting a comment on my post made by Tom Browder here, since I think it covers an important topic that is worthy of wider discussion:

December 16, 2022 at 1:45 am
Great article pair, thanks. But, according to my reading of the docs in “Object orientation,” a class created using has $attr is the same as has $!attr, i.e., private attributes; then the docs get a little muddy to me in the next paragraph, which I think needs a bit of work.

Now, I checked the docs and (not surprisingly) Tom is right. I also checked the repl and yes, it matches the docs.

BUT when I learned raku (from Think Raku) there were only two options:

has $.x;   # public attribute
has $!y;   # private attribute

So, it seems to me that the un-twigiled variant has $z; # private attribute has crept in without my noticing.

So, why am I agitated? The way I see it, raku has two gears (at least) for OO:

First gear is for "making the easy stuff easy". Just like in languages such as Python, you use the dot twigil (.), get all the accessors you want for free and can then use dot notation to access the attr.

class C {
    has $.x = 42;
}

my $c = C.new;
say $c.x;  # 42
$c.x = 666;
say $c.x;  # 666

Second gear is for "making the hard stuff possible". Just like in languages such as Java and C++, use the bang twigil (!) and this attribute requires getter/setter methods to be employed and is a key tool to enforce encapsulation.

class D {
    has $!y = 42;

    multi method y           { $!y }          #getter
    multi method y( $value ) { $!y = $value } #setter
}

my $d = D.new;
say $d.y;           # 42
$d.y = 666;         # error - need a proxy for this
$d.y: 666;
say $d.y;           # 666

[Larry did many good things - but this use of the colon to create a cognitive triplet of =, := and : was IMO the best of the lot]

So, IN MY OPNION, the third way of the untwigil is a MISSTEP in language features since it is at the same time easier to write and harder to use (than the dot twigil). If this was consistent, then the easy thing to write has $z; should also be the easy thing to use by being bound to the dot twigil variant. But then IMO that makes it too easy to get stuck in gear one.

A further negative to the untwigil is that the symmetric options $. and $! are easy to remember and impossible to forget, adding a third option creates more complexity for the coder and the maintainer.

So I have two questions:

1. Is the untwigiled variant now part of the language?
2. If yes, then do we think it SHOULD be in the language?

[librasteve]

Here's what jnthn said...

Is the untwigiled variant now part of the language? Now? :-) I believe has been for a decade or longer. It's mentioned in S12, and Rakudo implements it as described there (has $foo actually declares has $!foo and then makes $foo an alias to it).

If yes, then do we think it SHOULD be in the language? It's tempting to point out that it's a mild nuisance for the compiler to handle, but by that reasoning a lot of the language would be on the chopping block. :-)

I can't find mention of it now, but I think the syntax was once described as a sop to those who don't like twigils. However, I suspect the number of folks who are consider twigils on attributes their only dealbreaker for using Raku is pretty small - if nothing else because I can't remember the last time I saw this form in the wild!

[librasteve]

Here's what 2colours said...

I think from repo purism point of view, this issue might have a better place at Raku/Problem-solving.

Anyway, my two cents: if something already works with really low "maintenance cost", think twice before removing it. Raku is not a minimalistic language either way and we cannot have a discourse with potential users in the future who could miss it.

[librasteve]

Here's what lizmat said...

This would go through a deprecation cycle, so it won't be gone until at least 6.f or 6.g, or whatever we will call it then.

[raiph]

IN MY OPINION, the third way of the untwigil is a MISSTEP in language features since it is at the same time easier to write and harder to use (than the dot twigil).

To give you food for thought, I'll advocate in the reverse direction, in a manner I think is sympathetic to what Larry intended, or at least what I think he was being sympathetic toward.

---

Imo it's great design.

Sigils are easier to get than twigils.

A sigil'd attribute is easier to use, much easier, than a twigil'd one, whether it's a dot or an exclamation point one.

Here's my elaboration of that opinion.

First, twigils build on sigils. You have to know what a sigil is to learn about twigils. And if you learn very basic Raku coding, you will learn about sigils near the start, quite plausibly long before you learn twigils.

Second, the step between the following two is merely the declarator:

my $foo = 42;
has $bar = 99;

A newbie can focus on the distinction between my and has without getting distracted by a tsunami of other premature details ("Twigils?").

Third, the twigil free version is perfect for learning two big OO things: A) some preliminary basics, and B) the underpinnings of the single most important thing about Raku OO which is that it's all built on 100% encapsulation, something many other OO PLs get wrong. The importance of this cannot be overstated. And the fact this foundation, a necessary ingredient of data race free concurrency, aligns perfectly with not needing twigils, is deliberate I think.

Fourth, one can initialize attributes at object instance creation time without introducing twigils:

class foo { has $bar is built }

say foo.new: bar => 99; # foo.new(bar => 99)

An obvious dialog will arise next:

Newbie: Can I get the value of the $bar attribute from outside the class?

Guide: Sure. One simple and popular but potentially problematic way is via "dot" accessors.

And that is when you begin to step into the world of twigils.

It's a very good thing it is delayed until then. One of the most evil things in OO is setters, with getters a step on the slippery slope toward them. They are so enticing that folk love them, but the cost of making them too easy is well known by PL designers. cf Jonathan in an SO answer:

getters and setters are methods that have an implicit connection with the state. While we might claim we're achieving data hiding because we're calling a method, not accessing state directly, my experience is that we quickly end up at a place where outside code is making sequences of setter calls to achieve an operation - which is a form of the feature envy anti-pattern.

Yes, Raku makes dot accessors (getters/setters) easy to write if you must (which one really must for record types, but not objects in general), but Raku also makes sure they're not as easy to write as 100% private attributes, which really should be how coders write attributes in most cases.

---

The only downside I see, one that I suspect Larry did not foresee, is that someone influential on the official documentation was of like mind to you and avoided the way to teach good OO that I tried to outline above, instead unofficially deprecating twigil free attributes. One result is that most people currently learn Raku OO without ever hearing that twigil free attributes are a thing. Then, when they do, they're of no obvious value, perhaps even of negative value, because they've now learned in their mind to associate sigil'd variables with not being attributes. And then this becomes self-fulfilling. Not because twigil free attributes are worthy of deprecation, but because they've been unofficially deprecated (by documenters) which means they are currently kinda pointless despite being superb design.

Anyhoo, now for advocacy mode off.

---

One other reason to keep them, that isn't advocacy but just the way of things, and seems non-controversial even if their merit is, is that they already exist, and generally don't actively do harm merely by existing.

[2colours]

The only downside I see, one that I suspect Larry did not foresee, is that someone influential on the official documentation was of like mind to you and avoided the way to teach good OO that I tried to outline above, instead unofficially deprecating twigil free attributes.

This part really sounded like a bunch of (way too) big words so I went on to actually check what the documentation has about this to see if there was a deliberate act like that.

Turns out the starting assumption is wrong:

Alternatively, you can omit the twigil, which will still create the private attribute (with a ! twigil), and will also create an alias that binds the name (without the twigil) to that attribute. Thus, you can declare the same class above with

This is an insightful description of the situation actually:

class Foo1 { has $bar = 12; method test { say $bar, ' ', $bar.VAR.WHICH; say $!bar, ' ', $!bar.VAR.WHICH; } }
Foo1.new.test;
# returns 12 and the same Scalar for both
class Foo2 { has $!bar = 12; method test { say $bar, ' ', $bar.VAR.WHICH; say $!bar, ' ', $!bar.VAR.WHICH; } }
# Error while compiling: Variable '$bar' is not declared.  Did you mean '$!bar'?

So it is documented and it describes the actual behavior, as proposed in the Synopses.

By the way, this makes twigilless attributes look even more like "low fat syntax sugar". I really don't get the sentiment that this would be some revolutionary missed opportunity in teaching Raku OO - neither "missed" nor "revolutionary". People just don't use it, it's not really a "living meme" within the community.

Having said that, I'm still not sure if there can't be a good argument for twigilless attributes... although now that it became clear that it's just sugar, it's somewhat harder to imagine. 😅

[librasteve]

@raiph - thank you for your patience and wisdom - I agree.

[lizmat]

The only downside I see, one that I suspect Larry did not foresee, is that someone influential on the official documentation was of like mind to you and avoided the way to teach good OO that I tried to outline above, instead unofficially deprecating twigil free attributes. One result is that most people currently learn Raku OO without ever hearing that twigil free attributes are a thing. Then, when they do, they're of no obvious value, perhaps even of negative value, because they've now learned in their mind to associate sigil'd variables with not being attributes. And then this becomes self-fulfilling. Not because twigil free attributes are worthy of deprecation, but because they've been unofficially deprecated (by documenters) which means they are currently kinda pointless despite being superb design.

One of the reasons for the re-design of Perl, was that teaching it was troublesome. Especially wrt the behaviour of sigils. And I think Larry made a mistake here (again).

class A {
    has $foo;
    method bar {
        my $foo;
# lots of code
        $foo = 42;  # unclear whether this is the attribute or the lexical
    }
}

You shouldn't have to look back through your code to know whether it is an attribute or a variable.

A similar situation exists with formal parameters in blocks, where the twigil is also optional if there is at least one instance of use. Which causes all sorts of weird issues:

$ raku -e '{ my $a = $^a }'   
===SORRY!=== Error while compiling -e
Redeclaration of symbol '$^a' as a placeholder parameter.

So I think we need to be consistent, and call a spade a spade. That will allow teaching of twigils, when they're being used as first class citizens, not as something that can be ignored in some cases.

Therefore I'm re-opening this issue.

[codesections]

I've been giving this issue a fair bit of thought, of and on, and I'm not sure which design decision seems correct to me. I personally find the style used in the docs (always use twigils) much more readable, but I can also see some merit in @raiph's argument that the twigil-less style provides an easier onramp for some users.

But, even though I'm not sure of the best design, I strongly agree with one point @lizmat made: "You shouldn't have to look back through your code to know whether it is an attribute or a variable." In fact, the way Raku makes it easy to reason locally is (normally!) one of the things I like most about Raku.

So the fact that these two snipits produce the output they do really sucks:

class A {
    has $foo = "an attribute";
    method bar {
        # lots of code
        say $foo.uc; 
    }
}

A.new.bar;  # OUTPUT «AN ATTRIBUTE»

and

class A {
    has $foo = "an attribute";
    method bar {
        my $foo = "a variable";
        # lots of code
        say $foo.uc; 
    }
}

A.new.bar; # OUTPUT «A VARIABLE»

However, after mulling it over, I'm not convinced that this shows a design flaw with non-twigiled attributes – I think it might just be a bug. As @lizmat also pointed out, the design allows for a similar issue with placeholder parameters but Raku won't let you use them in confusing ways:

say { my $a = "a variable";
      $^a.uc
    }("an argument")
# OUTPUT: ===SORRY!=== Error while compiling 
# Redeclaration of symbol '$^a' as a placeholder parameter.

(@lizmat I wasn't quite sure if you liked or disliked this error message; you showed it as an example of "weird issues". But, for what it's worth, I think it's vastly better than letting you use $a after declaring both my $a and $^a.)

So, bottom line: I think that, regardless of what design decision we reach for future Raku versions, right now a class with a $foo Attribute and my $foo variable in one of its methods ought to throw a redeclaration error (edit: warning, see discussion in following comments). And IMO, not throwing that error warning is currently a Rakudo bug, not a design decision. Any disagreement?

If not, I guess the next step is to open that as a Rakudo issue – which won't resolve the longer-term design decision, but will at least avoid some of the worst consequences of the current design.

[jnthn]

So, bottom line: I think that, regardless of what design decision we reach for future Raku versions, right now a class with a $foo Attribute and my $foo variable in one of its methods ought to throw a redeclaration error. And IMO, not throwing that error is currently a Rakudo bug, not a design decision. Any disagreement?

Generally, declaration of a lexically scoped name that exists in an outer scope is not an error in Raku; for example, sub foo() { my $x = 1; if blah() { my $x = 2; } } is entirely fine. So in that sense, it'd be inconsistent.

[codesections]

Generally, declaration of a lexically scoped name that exists in an outer scope is not an error in Raku

That's true. But I'd distinguish between declaring a lexically scoped name that exists in an outer scope ("variable shadowing") from declaring a lexically scoped name that's passed into the current scope.

Thus, as you point out, the following code is fine:

sub foo() { 
  my $x = 1;
  if True { my $x = 2}
}

But this code produces a warning:

sub foo($x is copy) {
   my $x = 2
}
foo 1;

(and it errors without is copy or is rw)

In that case, $x was passed in explicitly, but the same redeclaration warning comes up when $x is passed implicitly:

sub foo {
  $^x;
  my $x = 2;
}

In my view, a class's attributes are much closer to the parameter-passing case than to the variable-shadowing case. After all, calling C.new.foo is equivalent to calling foo with C.new as its first parameter.

Thus, in my view, the code

class C {
    has $x = 1;
    method foo {
        my $x = 2;
        $x
    }
}

ought to generate the same warning as you'd get from passing $x to foo (explicitly or implicitly).

(However, I mistakenly thought that the parameter-passing case generated an error. Since it only raises a warning, I withdraw my suggestion that the attribute case throw an error. It should just be a warning; I'll edit that into my previous comment.)

[jnthn]

I'd distinguish between declaring a lexically scoped name that exists in an outer scope ("variable shadowing") from declaring a lexically scoped name that's passed into the current scope.

The thing is that the lexical "alias" to the attribute really does exist in the outer scope - that is, the scope of the class body! One can observe this by trying to use it outside of a method:

$ raku -e 'class C { has $a; say $a; }'
===SORRY!=== Error while compiling -e
Variable $!a used where no 'self' is available
at -e:1
------> class C { has $a; say $a⏏; }

[codesections]

The thing is that the lexical "alias" to the attribute really does exist in the outer scope - that is, the scope of the class body!

Fair. But the parameter also exists in an outer scope, albeit a very small one: the lexical scope of the parameter list. That's why sub foo($x, $y = $x + 1) {…} works!

(Of course, it's a small lexical scope that most people don't think much about. But I'd argue that the same is true of using attributes is a class body outside a method)

[ab5tract]

I would argue that the general lack of usage of this feature demonstrates that it falls under YAGNI.

We don't do anyone any favors by introducing even more special cases of syntax (see the removal of new Foo style method dispatch for a good example of making progress on this front).

There is just so much to learn already with Raku that I think that @raiph is inverted in his analysis of what makes for a good on-ramp in teaching someone. Fuzzing up the distinction between attributes and "regular" variables can be considered harmful to developing an understanding of attributes in the first place.

Catering to newbies should not, in my opinion, involve any form of "you can do it this way, but no one actually ever does". In my experience, that's more a cruelty than it is a concession.

In the best case, it's just (yet another) curious corner of the language that an experienced developer has to know about. In the worst case, it's a genuine example that demonstrates the point that Raku is more of a write-only language than we might like to admit. It could very well be the straw that breaks the butterfly's wings for a user who is on the verge of giving up on Raku because of the sheer amount of curious corners that they have already encountered.

In other words, I profoundly disagree that it is good pedagogy to present multiple forms of OO syntax (again, see our meaningful rejection of indirect method calls).