protocols

[CeylonMigrationBot]

[@gavinking] ### Disclaimer

Here's an idea that is only half-formed, that might not work out, but that I think is worth capturing here because it might be able to be adapted to solve several problems at once.

FTR: I stole the word "protocol" from Smalltalk. But in Smalltalk protocols have no real semantics, they're just a way to present members of a type in the UI. In particular, you can't have two members with the same name in different protocols, which is the whole point of this proposal.

Background

Recently, there have arisen a couple of scenarios in which we've wanted to give a type or package multiple members with the same name:

• Java interop: Java has 3 namespaces, for types, methods, and fields, whereas Ceylon has just one, resulting in collisions in certain corner cases. @FroMage wanted to add some new \f thing to the language in order to provide disambiguation.
• For API evolution: @matejonnet argues that interface evolution is more difficult in Ceylon than in Java because we don't have overloading.

These scenarios got me thinking about whether we should provide a way to arbitrarily segment the namespace of a type or package into protocols.

Declaring protocols

Because of the typesafe nature of protocols, a protocol would need to be declared. We could provide a dedicated syntax for this, for example:

 shared protocol Equality;

But it seems to me that we could just as easily make protocols a special annotation type:

shared final protocol annotation class Equality() 
        satisfies OptionalAnnotation<Equality> {}
shared annotation Equality equality() => Equality();

Or, alternatively, we could have a single predefined protocol annotation, and just define the protocol as a type:

shared interface Equality() satisfies Protocol {}

Assigning a member to a protocol

Again, we could provide a special syntax for this, perhaps something like C++'s public:, protected:, private:. I quite like the look of this, for example:

shared abstract class Object() 
        extends Anything() {

    shared default String string =>
            className(this) + "@" + hash.string;

protocol Equality::

    shared formal Boolean equals(Object that);

    shared formal Integer hash;

}

But this is a whole new special purpose syntax, and so just using annotations feels much more regular to me:

shared abstract class Object() 
        extends Anything() {

    shared default String string =>
            className(this) + "@" + hash.string;

    shared formal equality Boolean equals(Object that);

    shared formal equality Integer hash;

}

Or, with a single protocol annotation:

shared abstract class Object() 
        extends Anything() {

    shared default String string =>
            className(this) + "@" + hash.string;

    protocol(`interface Equality`)
    shared formal  Boolean equals(Object that);

    protocol(`interface Equality`)
    shared formal Integer hash;

}

This last option is unfortunately a bit verbose, but perhaps there's something we can do to improve it.

Disambiguating member protocol

Now we need a syntax to select a member by name and protocol. To me, :: seems the most natural choice.

value hashCode = hasher.Equality::hash;

So far, this is, I suppose, enough to solve the problems related to Java interop. Now we can write stuff like this to solve the problems related to Java namespaces:

javaObject.JavaField::memberName

Though the rules for precisely which members of a Java type would go in the default protocol are slightly unclear. Only methods and classes? Only methods and classes that obey Ceylon's naming conventions?

Now for API evolution, something additional is needed.

Protocol selection

For API evolution, you would define a protocol for each revision of a module, for example Version1, Version2, etc.

Now we can have an API like this:

 shared object system {
     shared Integer now => ... ;
     shared version2 Instant now => Instant(now);
 }

By default, the following code would be well-typed:

Integer now = system.now;

But a client of Version2 would want to write:

Instant now = system.now;

So the client would need to specify an ordered list of protocols in the import, either at the module level:

import some.api "1.2" [Version3, Version2];

Or at the source level:

import some.api [Version3, Version2] { system }

The precise syntax and semantics for this are a bit up for grabs, for example, perhaps we need something like this:

import some.api "1.2" [Version3, Version2, ...];

To say something like "then the default protocol".

Feedback needed

Of course I have lots of doubts about this brand new concept. WDYT?

[Migrated from ceylon/ceylon-spec#833]

[CeylonMigrationBot]

[@gavinking] As an alternative to the above proposal, we could do something much simpler, just involving hackery with naming conventions and import aliases. We could make you write:

 shared object system {
     shared Integer now => ... ;
     shared Instant now_version2 => Instant(now);
 }

And then:

import some.api { *version2 }

This would automagically alias now_version2 to now.

Obviously this is much more adhoc, much less typesafe, and much more difficult to write tooling for. But at least it would establish a well-defined pattern, and provide the underlying model for how the model-loader treats the corner cases in Java interop.

[CeylonMigrationBot]

[@gavinking] Going back to the original proposal, with the option of a single protocol annotation, what I like about it is that a type could be its own protocol. Screw defining the Equality interface, all you would need to write is this:

shared abstract class Object() 
        extends Anything() {

    shared default String string =>
            className(this) + "@" + hash.string;

    protocol(`class Object`)
    shared formal  Boolean equals(Object that);

    protocol(`class Object`)
    shared formal Integer hash;

}

And write:

value hashCode = hasher.Object::hash;

[CeylonMigrationBot]

[@FroMage] This means I move all namespace issues to 1.1, right?

[CeylonMigrationBot]

[@FroMage] BTW: there's four namespaces: field, method, type and JavaBean property.

[CeylonMigrationBot]

[@gavinking]

This means I move all namespace issues to 1.1, right?

Well, if we think we're going to do this, then, yes.

BTW: there's four namespaces: field, method, type and JavaBean property.

Well there's an issue with that. We don't want you to have to qualify every typename by JavaType::, every method name by JavaMethod::, and every property name by JavaProperty::.

So we have to figure out a reasonable set of rules for under what circumstances a member winds up in the default namespace. Saying "every method or type that conforms to Ceylon naming conventions" gets us partway there, but we can still get collisions between methods and properties, so it's not a whole solution.

However, if there is a way to specify the lookup order of protocols, which is I guess what you already need for API evolution, then I guess saying that the lookup order is JavaType,JavaMethod,JavaProperty,JavaField would solve the problem. Of course, the choice between that lookup order, and the alternative lookup order of JavaType,JavaProperty,JavaMethod,JavaField is essentially arbitrary.

Still, at least this stuff gives us a model for how to think about the problem.

[CeylonMigrationBot]

[@FroMage] Just to make sure I got things right: this would essentially support overloading between attribute/method/type right? The model loader would name them all foo and the typechecker would resolve based on lookup order or user-specifier, right?

[CeylonMigrationBot]

[@gavinking] Right. The uniqueness constraint would no longer be name is unique but (protocol,name) is unique.

[CeylonMigrationBot]

[@FroMage] Then it sounds like the right solution for interop. No more black magic that doesn't work.

[CeylonMigrationBot]

[@FroMage] I would not call it prototype though, to me it's more similar to a view, kinda like in SQL views which give you different views of the same object.

[CeylonMigrationBot]

[@gavinking] "protocol", not "prototype". I don't think "view" is a good word for this at all. A protocol is just a namespace; it's not an abstraction or a technique for abstraction.

[CeylonMigrationBot]

[@FroMage] protocol is also a term that doesn't convey at all what this is about. Especially since it has nothing to do with the common definition of the term. It's very confusing. Named-scope or view for example are much closer analogies.

[CeylonMigrationBot]

[@FroMage] Call this a namespace then, it's much more intuitive than protocol.

[CeylonMigrationBot]

[@gavinking] I think it's pretty reasonable to call a related set of operations a "protocol". I don't especially object to "namespace", but the point is that this is a special kind of namespace. We already have the hierarchical namespace of module/package/type/member, and then we would have this separate orthogonal namespace which cuts across the existing namespace hierarchy.

[CeylonMigrationBot]

[@matejonnet] Lets see the complete workflow

we have module some.api 1.2

shared object system {
    shared Integer now => ... ;
}

than update it to some.api 1.3

shared object system {
    shared Integer now => ... ;     
    shared version1.3 Instant now => Instant(now);
}

or maybe better

shared object system {
    shared version1.2 Integer now => ... ;     
    shared Instant now => Instant(now);
}

1. developer get somehow notified (compiler warning?) about the new version of used module
2. developer updates the import declaration, at the module import level
3. idealy the existing code works with no modifications, while new APIs are available and there are warnings on old api usage

The ordered list of versions is used to enable/disable old(deprecated) api? Meaning developer have to define import some.api 1.3 [version1.2] to make available methods annotated with versoin1.2?

[CeylonMigrationBot]

[@gavinking]

The ordered list of versions is used to enable/disable old(deprecated) api?

No, it's used to enable new versions of the API.

[CeylonMigrationBot]

[@matejonnet] Enabling old/new version is the matter of convention right?

Wouldn't be better to annotate old, so that users are "reminded" that they have to update. They are already in the midle of migration to a new version by specifing new import module version.

Also once deprecated methods are removed, the code don't have to be changed to get rid of unnecesary version annotations.

[CeylonMigrationBot]

[@gavinking]

Enabling old/new version is the matter of convention right?

Yes, just a convention.

Wouldn't be better to annotate old, so that users are "reminded" that they have to update.

Well that way existing code breaks...

[CeylonMigrationBot]

[@matejonnet]

Well that way existing code breaks...

It can be solved at tool (IDE) level, by automaticaly offering to add old version annotaion while defining import.

[CeylonMigrationBot]

[@akberc] For API evolution, the overloading solution must not force the application developer to retrofit code to select a version of the evolving API. For the API evolution to be seamless to the developer who used an unversioned API that evolved later into V2, some random thoughts:

• API that evolves must be an interface with no overloading. The implementation problem then becomes a problem of set intersection, with the sets being the default/formal members of the interface supertypes of the implementation class.
• Currently, two methods of name m1 in supertypes may only be inherited if they share a common ancestor that had a single m1. In other words, the multiple m1s are actually one in the hierarchy and we are not really doing much magic.
• When no such intersection exists, instead of an error, set-name prefixing (explicit interface name) would be the natural choice for namespacing the discrete implementations of multiple m1s.
• I think the annotation actual then requires a list of optional disambiguating parameters -- the interface type -- but the annotation default can retain current behaviour and can only apply when there is a common ancestor. The definition of actual is: "Annotation to mark a member of a type as refining a member of ^a^ supertype." (emphasis added). So, there may be room in the spec to stretch it a bit.
• Java overloading may or may not fit this abstraction, but overloaded members may be visualized in a similar way, with the actual annotation taking a list of types of Java method parameters.

Something like this:

Given:

interface Api {
    shared formal String m1 (Integer i);
}

interface ApiV2 {
    shared formal Integer m1 (Integer i, String s);
}

Then,

class Impl() satisfies Api & ApiV2 {
    shared actual(`interface Api`)
    String m1(Integer i) => m1(i, "").string;

    shared actual(`interface ApiV2`)
    Integer m1(Integer i, String s) => 0;
}

--OR-- less desirably --

class Impl() satisfies Api & ApiV2 {
    shared actual
    String Api::m1(Integer i) => m1(i, "").string;

    shared actual
    Integer ApiV2::m1(Integer i, String s) => 0;
}