Semantics of Traits and Trait Composition

[apblack]

Semantics of Traits and Trait Composition

  During the Grace teleconference on 9th February 2016, I was asked to write down the semantics that I expect from traits and trait compositions in Grace.

Object Constructors

Everywhere in this document, when I write Object Constructor, I intend to include also the class shorthand, since this syntax is defined as equivalent to a method whose body is an object constructor.

Object constructors are unchanged by this proposal. Any object constructor without an explicit inherits statement implicitly inherits from graceObject.

We might want to introduce a way of eliminating the automatic inheritance, e.g., inherits nothing, where nothing is a pre-defined identifier.

Trait Constructors

I propose that we invent a syntax for trait constructors. I further propose that this be the same as the existing syntax for object constructors, except that it is preceded by the keyword trait, and that the body of the trait constructor be restricted to containing only uses statements, method definitions, type definitions, and comments; inherits statements, field declarations and top-level code are not allowed. [An alternative syntax using the annotation is trait on an object constructor isn’t legal, because trait is a keyword, and annotations require identifiers.]

Everywhere in this document, when I write Trait Constructor, I intend to include also the trait shorthand. Parallel to the class shorthand, the trait shorthand shall be defined as equivalent to a method whose body is a trait constructor.

Trait constructors can’t inherit, so there is no implicit inheritance of graceObject by a trait constructor.

Any object that satisfies the restrictions on traits (contains only methods and types) is defined to be a trait, and can be used by another trait or object. The purpose of the trait constructor is to provide error messages if a programmer inadvertently breaches the trait restrictions. Note that, as a consequence, graceObject is itself a trait, and can be used by an object that wishes to re-acquire one of the default behaviors.

Inheritance

I’m not proposing any changes to the way inheritance works as part of introducing traits. (I would like to see a drastic simplification of inheritance, but that is not part of this discussion.) Briefly, the way that I think inheritance works at present is as follows.

1. Any object constructor can contain an inherits statement that refers to a fresh object. This fresh object will become the superobject of the object generated by the constructor (hereinafter, the generated object). All attributes of the superobject will also become attributes of the generated object, with the same visibility (public, readable, or confidential), unless there is an overriding definition for that attribute.
2. An overriding definition means a definition written in the body of the object constructor.
3. The inherits statement can be modified by an alias clause, which defines additional names for one or more attributed of the superobject. These additional names also become attributes of the generated object. Question: should the default visibility of names introduced by alias be confidential? This is usually what one wants, but it’s inconsistent with the default for other methods.
4. The inherits statement can be modified by an exclude clause, which removes one or more names from the set of names that become attributes of the generated object.
5. Overriding definitions for types are not allowed.
6. Inheritance takes place in two phases. First comes the attribute collection phase, described above. Second comes the initialization phase, which is deferred until the attribute collection phase is complete. As a consequence, the generated object can change the semantics of the initialization of the superobject. It does this by overriding the definitions of one or more methods that the superobject uses during initialization.

   Trait Use

7. A trait constructor or object constructor can use one or more supertraits. A supertrait is denoted by an expression that returns a fresh object that qualifies as being a trait. It does not have to be defined using a trait constructor.
8. The supertraits in the uses clause are composed by summing the attributes of each of them. The sum operation is defined in Section 4 of S. Ducasse, O. Nierstrasz, N. Sch ̈arli, R. Wuyts, and A. P. Black. Traits: A mechanism for fine-grained reuse. ACM Trans. Program. Lang. Syst., 28(2):331–388, 2006. To briefly summarize:
   • The sum of a trait containing an attribute a and a trait containing a method b is a trait with attributes a and b, whose bodies are the bodies of a and b in the component traits.
   • The sum of a trait containing an attribute named a with another trait containing an attribute named a is a trait with an attribute named a whose body is the join of the bodies of the bodies of a in the composition. If the bodies are equal, then the join is that body; otherwise, the join is the distinguished value traitConflict. We can approximate equality conservatively; in particular, if the two bodies are parameterized, we can assume them to be different, even though the value of the parameter might be the same in both cases.
9. Confidential methods obtained from a trait remain confidential in the generated trait or object.
10. uses statements can be modified by alias and exclude clauses, just like inherits statements, and with the same effect.
11. After inherited and trait methods have been added to the generated trait, locally defined attributes are added. These will override attributes from a superobject or a super trait that have the same name.
12. Overriding definitions for types are not allowed.
13. Once the locally-defined attributes have been added, it is an error for the generated trait or object to contain a method whose body is traitConflict. Ideally, this error will be reported statically.
14. An object constructor that uses traits can also inherit. If it does so, the first phase of inheritance (attribute collection) takes place before any trait methods are included in the generated object, and the second phase of inheritance (initialization) takes place after all trait methods and local methods have been included, as described above.

    Default methods

Before traits, all objects by default inherited a few useful methods, notably =, ≠, ::, andasString. This remains true for object created by object constructors, but it’s no longer true for objects created by trait constructors. This means, for example, that you can’t put a trait in a dictionary (at least, not in the usual way.)

I don’t think that this is a big problem, since an object or trait constructor can only use a fresh trait, not one that’s been stored in a dictionary. But the loss of asString and the concomitant ability to print any object is a practical problem.

We could solve this by instead by putting the default methods into trait objects, but giving the uses clause the special property that if a component trait contains a method with one of the default names and the default body, then that method is excluded from the composition. (I think that this is kjx’s option 2 from issue 35.) I must admit that this is a kludge, but I’m concerned that not having asString will become a problem. Perhaps the best thing would be to try getting along without the kludge (so that traits don’t have an asString), and see how hard it bites.

[kjx]

So: I thought about this question last night. I really want to make progress and fast, and I’m on holiday until Monday with spotty internet access and little time to think. So - to make progress - I propose we adopt this. Then at least we have something we can specify, and know what to describe, and Andrew can implement it!

[kjx]

The sum operation is defined in Section 4 of S. Ducasse, O. Nierstrasz, N. Sch ̈arli, R. Wuyts, and A. P. Black. Traits: A mechanism for fine-grained reuse. ACM Trans. Program. Lang. Syst., 28(2):331–388, 2006

I haven't really looked in any depth - but another decisive advantage of this design is that we already have a formal semantics for it in that paper.

[kjx]

I don't want to re-open #38 because I see no other way to make progress. But I got a working Internet for a bit, so I recorded my remaining questions about this proposal at #39

[apblack]

Clarification: traits can capture state. Everything in Grace can capture state, so it would be very hard to exclude this case. What would the error message say?

[apblack]

An alternative creation myth

I came up with this in the shower this morning, after spending an hour or so implementing trait method collection and checking.

1. In the beginning was graceObject. It was gifted its method from the ultimate creator of all things.
2. All other objects inherit from graceObject, unless they specify some other object in an inherits statement.
3. Traits cannot have an inherits statement. (Therefore, they all inherit from graceObject.)
4. When collecting attributes for an object by climbing the inheritance chain, all methods are included (as modified by alias and exclude) up to and including graceObject, as well as those obtained from used traits.
5. When collecting attributes for an object by climbing the trait usage chain, all methods from all used traits are included (as modified by alias and exclude) except those from graceObject.

I think that this has the right effect, and has the advantage that we don't have to change our existing inheritance story.

[kjx]

I think this is OK - it's a variant on the "default methods are treated specially in traits" except we identify default methods by their being in graceObject rather than elsewhere.

and has the advantage that we don't have to change our existing inheritance story.

not sure quite what you mean by "don't have to change". It's a tweak, but not a very big one. What I like about this is that object { } is fundamental, and (almost) everything else can be defined in terms of it (except I guess graceObject?)

[kjx]

This also relates to Nixonianism #59 and default methods (obviously) #49

notably, there are a range of possibilities (this text also in #49)

1. objects don't have any default methods: trying to call them leads to a dynamic type error
2. objects have default methods but the object is obviously incomplete (a direct instance of a trait or an abstract class) #59, #56
3. objects have default methods but we know they can obviously never work (they are abstract or trait conflicts or ... #59) so calling them leads to a dynamic type error
4. objects have default methods that non-obviously never work (a template method than calls another method that is abstract or trait conflict) and calling them leads to a dynamic type error
5. objects have default methods but the object is uncooked #56 so calling them probably leads to a variable uninitialised error
6. objects have default methods that just (sometimes) break.

One consequence of this is that any systems code, particularly in debugging etc, can never really depend on default methods: this kind of defensive coding is always required: https://github.com/gracelang/language/issues/39#issuecomment-183551871

[kjx]

this seems to be waiting on #47 (manifest) #49 (default methods) in particular and a bunch of other ones...

[kjx]

(where I've got to)

traits and default methods: the problem:

EITHER traits cannot have default methods

OR we have to complicate the already crazy inheritance/trait “algorithim”, picking one of:

• handle default method bodies specially
• handle the trait defining all default methods specially
• add in a general body meet “merging rule” like the Smalltalk Traits.

Note that handling default methods based on their names doesn't work, neither does class (or trait priority) -- because e.g. you might have either traits or classes either using or requiring default methods.

I think the inheritance rules #67 are already very complex. I don't want to add in more special cases. So I think traits should not have any default methods. This is less of a problem than you might think, because you cannot trust default methods anyway.

how do avoid traits getting default methods

OPTION 1: no object constructor gets default methods; class shortcut somehow adds them in

• e.g. (simpler, 1a) inherits from a superclass that provides default method if there is no explicit inheritance
• (more complex b) uses a trait that provides default methods if not (or not all) default methods are defined

OPTION 2:

• every object constructor gets default methods (must now be a trait)
• "inherit done" turns default methods off
• the traits shortcut adds in "inherit done"...

In other words, it seems programming with objects has to have a small bias towards either making traits or making classes...

[KimBruce]

Traits should not have default methods, objects (and hence classes) should. Traits just have methods (or perhaps a very little more) Objects are traits with defs and vars, initialization code, and default methods.

[apblack]

What's wrong with the rules that I set out above? I've implemented them, and they seem to work.

Doubtless I've missed something. Can you tell me what it is, before we start on this all over again?

[kjx]

What's wrong with the rules that I set out above?

Those rules "handles the traits definition the default methods specially" (4,5), and also as "graceObject" which is special (1). Plus "uses" seems to work differently from "inherits" at least some of the time. Re-reading, I'm not sure how _"When... climbing the inheritance chain, all methods are included as well as those obtained from used traits" is modified by "trait usage chain, all methods from all used traits are included. except those from graceObject."_ Perhaps none of these points are important.

I've implemented them, and they seem to work.

I'm in no way proposing you should change anything you've implemented in the short term (i.e. before your course runs)

[kjx]

Kim - so do you mean you'd be happy with OPTION 2 above?

if not, we're back to Andrew's original four-part proposal: either traits cannot be objects (so must be something else); or raw traits cannot be "instantiated"; or the rules get more complex and assymetric

[apblack]

Yes, those rules handle inherits and uses differently. I view that as a feature of uses — that it behave differently from inherits. (If it behaved the same, why on earth would we have two keywords?)

And yes, graceObject's methods are treated specially. I'm not sure that's necessary, but it seems to be what programmers will want most of the time.

Having default methods is a compromise. I agree that the language would be purer, and simpler, if we didn't have them. But they make programming easier for novices.

It's the same story with having different default visibilities for methods, defs, and types (what is the visibility of a type?). They complicate the language, but make programs simpler.

In the end, it's a matter of taste how far one goes with this. I think that you have gone overboard by saying that alias also changes visibility (but that may be because I don't yet know how to implement the change in visibility). But that does not mean that there is a "theorem" that traits can't have default methods.

[kjx]

it's a matter of taste how far one goes with this

absolutely. the problem is we have different tastes :-)

(If it behaved the same, why on earth would be have two keywords?)

because we want to distinguish between traits and classes, and these two keywords help us make that distinction (i.e. the big distinction is you get only one inherit clauses and only that inherit clause can inherit from a class). That distinction still holds even if the underlying semantics are the same.

Having default methods is a compromise. I agree that the language would be purer, and simpler, if we didn't have them. But they make programming easier for novices.

On default methods at least I agree that we need them: I'm trying to find the smallest tweak to the language that gets them.

funnily enough, talking about this stuff for the other project with Mark & Sophia

saying that alias also changes visibility

I know Kim likes this, but I'm... agnostic.

[kjx]

But that does not mean that there is a "theorem" that traits can't have default methods.

I'm not saying that there is a "theorem" that traits can't have default methods; I'm saying the coherent design options seem to be that either traits cannot have default methods, or that we have to complicate (an admittedly already complex) part of the language.

Flattening #71 has been important to me since the Pomona meeting, and if anything it has got more important to me since then. What I'm after (in chasing flattening) is as simple as possible a description of what inheritance & trait composition actually does: I think they are sufficiently tangled so they have to be treated together. To me, aliasAndAbstract makes vars & defs simpler to explain than our current collection of alias and exclude; making uses have the same as semantics as inherits; not having special cases in precedence etc (although of course I still want MY special cases in the parser)

In the end, it's a matter of taste

yes, and it's only after re-reading the TOPLAS paper this week that e.g. I realised how close our current design is to the Smalltalk traits design - and not e.g. a simplification of it.

[apblack]

yes, and it's only after re-reading the TOPLAS paper this week that e.g. I realised how close our current design is to the Smalltalk traits design - and not e.g. a simplification of it.

I think that's because you can't simplify it any more and still make it useful. Almost all other trait implementation make it much more complicated!

One simplification that we have made is to eliminate super from inheritance, and use the same alias & exclude mechanism for both inheritance and trait usage.

Let's recall how we got to the above treatment of default methods. A class isn't something that uses the class syntax — it's a method that tail-returns a fresh object. Similarly, a trait isn't something that uses the trait syntax, it's a class that happens to obey the "tritely" restrictions. So we can't make the existence of default methods depend on the use of the trait or class syntax — that won't do what we need.

The alternative "creation myth" that came to me in a vision two weeks ago now, and (I think!) have implemented in minigrace, is that whether you get the default methods depends on whether you use uses or inherits. I think that this works, but we won't really know until we have programmed with it a bit. That's what I want to do now — stop talking about this and get down to writing code using it.

[kjx]

I wrote some stuff in the spec about default methods. https://github.com/gracelang/language/blob/portland-james/spec.md#default-methods

The spec now says:

the objects created by the class syntax inherit from graceObject if no inherits clause is supplied (but not objects created by the trait syntax or by object constructors

and

Some objects, notably instances of raw traits and done do not conform to Object

rather than

Notice that the public methods implicitly inherited from Object are implicitly included in all types.

https://github.com/gracelang/language/blob/portland-james/spec.md#type-object

[apblack]

The above links to the spec no longer work. Here is one that does: http://web.cecs.pdx.edu/~black/OOP/GraceResources/spec.html#type-object

I've updated the spec (version 0.7.3) so that the section on Type Object and the section on default methods are consistent. So I'm closing this issue.