static methods and attributes

[gavinking]

So, I'm throwing this one out there to see what ya'll think.

A long time ago I decided that Ceylon would not have static methods. The reason for this was that I always hated them in Java: in order to declare a regular function, you were forced to define a rubbish class with no instances, just to stick the function on it, and then to call that function, you had to do something nastily verbose like Integer.parseInt(). These functions had nothing to do with the schema of any type, and they don't have access to the instance-level state of the class, they are just stuck onto the side of a class for no good reason.

Some things happened since that time to make me want to reevaluate this:

1. We had to anyway build almost everything needed to support static methods into the typechecker in order to support Java interop.
2. We added constructors, which sorta look like static methods. In particular, they are invoked on a class name, instead of an instance.
3. People started asking for "factory functions", which are, essentially, a particular sort of static method.
4. I figured out how to resolve the "don't have access to the instance-level state" conundrum: simply require that static members be declared at the beginning of a class with constructors, before any instance-level state is allocated.
5. I started writing lots of code that interoperates Java with Ceylon and found it sorta weird that I was totally comfortable calling Java static methods in Ceylon, but couldn't actually write a static method in Ceylon.
6. I started wanting to define a constant values shared between all instances of a class, without polluting the package namespace.
7. I ran into a (anti-)pattern used in IntelliJ APIs where a service implementation is supposed to define a certain specially-named static member. I couldn't do this in Ceylon, so this class had to be written in Java.
8. I realized that there actually is one very good motivation for static: static members can bypass the visibility restrictions that are imposed on toplevel functions, and access private state of instances of the class. This is unambiguously useful, especially since we don't (yet) have package-private.

Thus, at this point, it would be sooo easy to add static methods and attributes (it basically amounts to one new annotation), that it's hard to see why we shouldn't. It would look like this:

class Integer {

    shared static Integer|SyntaxError parse(String string) => internalParse(string);
    shared static Integer sum({Integer*} integers) => integers.fold(0)((s,x)=>s+x);

    shared new (Integer i) { ... }

    ...

}

Note that I don't see why interfaces couldn't have static members. So this need not just be for classes.

So, pros and cons of doing this include:

Pros:

• it makes it slightly easier to port Java code to Ceylon (paste-Java-as-Ceylon will work a little better)
• it results in the more-expressive Integer.parse(text) and Integer.sum(totals) instead of parseInteger(text) and integerSum(totals)
• it's easier to "discover" Integer.parse() and Integer.sum() in the IDE than it is to discover parseInteger() today
• it provides a nice way to share private state/operations between a class and functions which operate on its instances,
• it provides a way to define class-local constants,
• it probably resolves the perceived need for "factory functions"
• it makes constructors look less "irregular", since static members and constructors share a essentially the same rules
• it slightly eases the cognitive tax of programming in mixed Java/Ceylon projects, since that's one more construct that maps across cleanly (ditto for JavaScript where you have stuff like Math.xxxx)
• it might make some things from Dart and Typescript easier to represent @jvasileff @lucaswerkmeister wdyt?

Cons:

• it might encourage Java-isms of creating rubbish classes just to stick functions on them (but it's easy enough to tell people to use object instead of class for that
• the people (person?) who think(s) that introducing constructors were a terrible mistake that makes the language less regular might perceive this as doubling down on that irregularity rather than resolving it by make it less irregular
• it's an additional language feature that is clearly not a critical thing we have to have
• it might be more difficult to implement in Dart and JS than it is in Java (I doubt this)
• I would have to admit I was wrong about something

Thoughts?

[gavinking]

So now we have three options for “enums” (toplevel object, case constructor, static object member)?

Yes.

But, of course, they're all the same thing. object is syntax sugar for a value constructor, according to the spec.

[gavinking]

One more bug: the generated methods for defaulted parameters don't get a static annotation generated. Need to fix that.

Done.

[tombentley]

ceylon.language::Integer.name (the constructor parameter) now has qualified name ceylon.language::Integer.null.integer in your branch @gavinking. It used to be ceylon.language::Integer.integer

[gavinking]

@tombentley are you sure that isn't due to @d2c0d47f4591c48d1c88e50f1859b4247cad47e8 which is actually on master?

[tombentley]

Probably it is due to that.

[gavinking]

So what should it be? It seems to me that both of those are wrong. How about ceylon.language::Integer..integer? Or ceylon.language::Integer.new.integer?

[tombentley]

I think I prefer the new one, because it makes me think "ah, it's the constructor parameter" where as .. doesn't make me think "constructor" (at least, not immediately).

[jvasileff]

That's ambiguous:

class Integer {
    shared new(Integer integer) {}
    shared new \inew(Integer integer) {}
}

[jvasileff]

BTW, I know qualified names aren't guaranteed to be unambiguous, but for runtime models, they can be made unambiguous by adding qualifiers. Using new would make that task trickier and more complicated. So I'd prefer ceylon.language::Integer..integer.

[gavinking]

@tombentley For this code:

shared void run() {
    print(Integer.parse {
        string = "1235";
    });
}

I got this error:

Error:(3, 9) ceylon: Ceylon backend error: long cannot be dereferenced
Error:(2, 11) ceylon: Ceylon backend error: method invoked with incorrect number of arguments; expected 1, found 0

[gavinking]

@tombentley the generated code is long arg$0$1 = long.parse$radix(arg$0$0);

Should be Integer.parse$radix(arg$0$0);.

[ePaul]

Just for understanding ... I would expect that from the outside (i.e. for a caller), a static method (with return type of the class itself) and a named constructor are indistinguishable from each other, and you can transform one into the other without breaking any clients.

Is this the case?

[quintesse]

Is this the case?

I'm not the one to answer your question, I just think they might not be the same because if we have a type that implements Destroyable we can use it in try resources like this:

try (foo = SomethingDestroyable()) {
    // use foo here
}

But the rule is that you can only do this with object instantiations. So if static methods would be the same as named constructors it would mean you could do:

try (foo = SomethingDestroyable.named()) {
    // use foo here
}

and it could possibly be something that's not an object instantiation. So that would be somewhat ambiguous IMO.

... ok, while writing this I decided to try this out and it turns out you cannot use named constructors (for objects implementing Destroyable) in try-resource statements! That's surprising. Is that on purpose @gavinking ?

[jvasileff]

Assuming you should be able to use named constructors for Destroyables, the only advantages that constructors have over static factories that I can think of:

• constructors can be extended
• constructors can provide Destroyables for try with resource

The Destroyable limitation for static methods could become a frustrating limitation. Should there be an exception to allow this usage? It could even be limited to static methods that have a return type matching the containing classOrInterface.

[lucaswerkmeister]

One of the cited usecases for static methods is that they can return cached instances, which is precisely why they shouldn’t be allowed to provide Destroyable resources for try-with-resources IMO.

[jvasileff]

I doubt things you would cache would be Destroyable.

[quintesse]

which is precisely why they shouldn’t be allowed to provide Destroyable resources for try-with-resources IMO

I always found that too heavy-handed, because now there's no way for me to use factory methods for my destroyables.

I did just think about something that could perhaps solve that issue: how about allowing methods that return constructors inside a try-resources? That way you can at least have a factory method choose the kind of class to create while still keeping the construction itself inside the try.

[gavinking]

@ePaul

Just for understanding ... I would expect that from the outside (i.e. for a caller), a static method (with return type of the class itself) and a named constructor are indistinguishable from each other, and you can transform one into the other without breaking any clients.

That is the eventual goal, yes, exactly.

Is this the case?

It is not the case right now. But it's planned.

[gavinking]

@quintesse

So if static methods would be the same as named constructors it would mean you could do:

Well that's a different special case. I think we're more talking about binary compatibility for regular constructors. Not this special case.

That's surprising. Is that on purpose @gavinking ?

No, I would say it's a bug. Please open an issue.

[gavinking]

@lucaswerkmeister

One of the cited usecases for static methods is that they can return cached instances, which is precisely why they shouldn’t be allowed to provide Destroyable resources for try-with-resources IMO.

Exactly.

[gavinking]

@jvasileff

Assuming you should be able to use named constructors for Destroyables, the only advantages that constructors have over static factories that I can think of:

• constructors can be extended
• constructors can provide Destroyables for try with resource

Huh? This isn't true at all. There's all kinds of things a constructor can do that a static method can't do. Set final fields, for example.

[jvasileff]

Huh? This isn't true at all. There's all kinds of things a constructor can do that a static method can't do. Set final fields, for example.

Come on Gavin, of course it would delegate to a possibly non-shared constructor (or possibly a subclass constructor) to actually create the instance. But then it can do more stuff constructors can't.

[jvasileff]

Exactly.

If the author the static function File.newFile() returns a cached File that also satisfies Destroyable, we've got problems.

[gavinking]

If the author the static function File.newFile() returns a cached File that also satisfies Destroyable, we've got problems.

Right, which is why it should be disallowed.

[jvasileff]

Right, which is why it should be disallowed

pffff no, we should not use that developer's stuff. I'm sure they also would screw up the newFile constructor.

(of course I meant TempFile or Reader or whatever)

[quintesse]

pffff no, we should not use that developer's stuff. I'm sure they also would screw up the newFile constructor.

Right, that's why I found this restriction so heavy-handed. It disallows things that are perfectly fine. And I can call Destroyable.destroy() manually anyway, so it's not that hard to do things wrong.

[lucaswerkmeister]

But if you think you know better, you can also do this:

MyDestroyable res = MyDestroyable.staticMethod();
try {
    // ...
} finally {
    res.destroy();
}

I think given the contract of Destroyable (and the fact that Obtainable exists), requiring a provably new Destroyable object in try-with-resources is a very reasonable restriction, and not one we need to relax.

pffff no, we should not use that developer's stuff.

That sounds like C logic to me. Let’s give everyone a mile of rope. If they shoot themselves in the foot, it must be their fault.

[gavinking]

I think given the contract of Destroyable (and the fact that Obtainable exists)

Exactly. We already introduced Obtainable for precisely this purpose so I don't even know what we're arguing about here.

[jvasileff]

This isn't a mile of rope. It's a very specific scenario where a static factory method of the class Reader might return a Reader. The author of Reader should be aware of how Destroyable works.

I believe the reasonable counter argument would be if Reader has non-factory static methods that return Reader, and such functions would be common and potentially confusing. But no-one has made that argument yet.

Regardless, it seems @gavinking already agreed that static factory methods should be usable in try-with-resource when he wrote:

It is not the case right now. But it's planned

[lucaswerkmeister]

I believe the reasonable counter argument would be if Reader has non-factory static methods that return Reader, and such functions would be common and potentially confusing. But no-one has made that argument yet.

I haven’t really seen an argument that factories for Destroyables would be useful either, so I couldn’t say which one is more likely to turn up. It just sounds weird to me.

[chochos]

are there going to be any changes to the metamodel because of this, or is it going to stay the same? I'm asking because you still need to bind attributes to instances to get their values, and pass an instance to methods in order to call them, when really you shouldn't have to, since they're static...

[chochos]

OTOH a whole new set of metamodel types for static stuff is going to be hell to implement

[gavinking]

Ouch, OK, @tombentley, I have run into a tricky problem. Consider:

class Outer<T> {
    shared static interface Interface {
        shared formal T get();
    }
    shared static abstract class Class() {
        shared formal T get();
    }

    shared new () {}
}

• The static interface is fine, because the compiler creates a generic toplevel interface Outer$Interface<T>.
• The static class is broken, because the compiler creates an inner class with no type parameters, and Java doesn't consider that the type parameters of the outer class are in scope.

So we need to either do the same thing to Class that we do to Interface, or we need to at least copy the type parameters of Outer to Class.

What a pain.

[gavinking]

@chochos I imagine we can just treat static members like toplevels in the metamodel.

[jvasileff]

@lucaswerkmeister some of the reasons static factory methods are useful: https://github.com/ceylon/ceylon/issues/6515#issuecomment-248332422

Of those, returning an object or cached instance is much less likely for Destroyables, but even that's not unimaginable for something like /dev/null, and empty resource, or even a random number generator.

[quintesse]

But if you think you know better, you can also do this:

To me that just shows that it's so laughably easy to step around the restriction that you could just as well have no restriction. And it will only cause weird things when a developer decides to create factory functions and then has to tell their users "sorry, you can't use try-resources because the developers of the language decided they knew better, just use try-finally". We're just taking away the candy, we're not creating any real security.

[ePaul]

I'm not sure this is a good idea, but maybe one could have some annotation which means "this function is creating (and returning) a new object" (meaning an object which did not exist before the call to the function). The compiler would ensure that it in fact does this, by making sure the returned object is either coming from an instance creation expression (i.e. using a class name as a function), or from another function which is annotated this way.

Named constructors would have it automatically, and all function calls of functions with this annotation are allowed in a try-with-resources for destroyable objects.

[ePaul]

@gavinking in Java, static nested classes (and interfaces) don't inherit any type parameters from their outer classes/interfaces – because the nested classes object's are not bound to an outer instance, they also don't have access to the type parameter. If I understand your last comment right, you want the behavior be different for Ceylon, so nested static classes automatically have the same type parameters as the outer ones? That sounds like it would cause trouble.

[gavinking]

@ePaul When I write:

class Class<T> {
    static interface Interface {
    }
}

It's clear from the block structure of the language that T is in scope within the body of Interface, and so I should be able to declare a method that returns T. This is nothing to do with anything about having access to an instance of `Class. Type parameters aren't instance variables.

nested static classes automatically have the same type parameters as the outer ones?

Well, not quite: they can declare additional type parameters.

That sounds like it would cause trouble.

I don't see why.

[gavinking]

Note that the Ceylon typechecker already prevents me from writing Class.Interface anywhere, and requires me to write Class<String>.Interface.

[ePaul]

Ah, okay. So when we have a type declaration inside of a parameterized type, we actually have one type for each of the possible (type) instantiations of the outer type, and Class<String>.Interface is a different type than Class<Integer>.Interface. I guess I missed that when reading the spec previously – it is actually not specific to static inner types.

Thanks.

(I guess we could now define Map<Key,Value>.Entry as an alias for Entry<Key,Value>, or the other way around.)

[jvasileff]

This seems to work ok for Map.Entry (as long as you don't want to import Entry), but I think generally diminishes the value of static members for generic types since, by the nature of static members, the containing type's type parameters are often irrelevant to or even misleading for the static member.

But if it will continue to be valid to use them, and if they'll be required, a few questions:

1) Will static objects and values of generic types be disallowed? 2) Will type argument inference work for providing arguments for the qualifying type? 3) Will the normal variance rules for the use of type parameters be relaxed?

[gavinking]

1) Will static objects and values of generic types be disallowed?

This is a very good question. I suppose so, yes.

2) Will type argument inference work for providing arguments for the qualifying type?

It already does, doesn't it?

3) Will the normal variance rules for the use of type parameters be relaxed?

WDYM?

[gavinking]

By the way, I think these arguments against considering type parameters in scope in a static are pretty unmoored from the actual use cases here. Consider the case of factory methods, e.g. Array<String>.ofSize(). If we want factory methods to look like constructors, then the type parameter goes in the instantiated type, no on the factory method.

[jvasileff]

I suppose it's equally nice to have static factories that require the same TAs as the containing type look like constructors as it is ugly to have to make up TAs for factories and other members that don't need them.

[jvasileff]

Ok, yes, it looks like TA inference already works when calling a static method. But on the other, this doesn't compile:

shared class Box<out T> {
    shared static Box<T> create(T t) => Box.make(t);
    new make(T t) {}
}

[gavinking]

But on the other, this doesn't compile:

Ah right, we can relax that check for statics since they aren't part of the schema of the type.

[gavinking]

1) Will static objects and values of generic types be disallowed?

Done.

3) Will the normal variance rules for the use of type parameters be relaxed?

Done.

I think I've done the right thing with these.

[ePaul]

Some philosophy:

static members of a type don't belong to the instances (of course), but conceptually can be seen as instance members of a "type object".

Parameterized types are actually families of related types, thus there is one "type object" for each parameterization: List<String> and List<Integer> are separate types, and thus have their own static members (even if they all have the same implementation). As there is no way to initialize values for each member of those families, disallowing static objects and static values makes sense.

On the other hand, this still leaves open the need for static members for the whole type family (as opposed for each member).

For example, I could imagine this (I know that Ceylon doesn't need a Comparator interface, but if it had one it could look like this):

shared interface Comparator<in Item> {
   shared Comparison compare(Item left, Item right);

   shared static Comparator<Comp> increasing<Comp>()
       given Comp satisfies Comparable<Comp>
     => object satisfies Comparator<Comp> {
            compare(left, right) => left.compare(right);
        }
   ...
}

This function would not belong to a Comparator<X> for arbitrary X, but to the whole family, and it has its own type parameter.

(Alternatively, it could be thought to belong to just to some subset of the members of the family, those ones whose type parameter implements Comparable).

[lucaswerkmeister]

I’m still not sure how this would play together with type classes (#4005). Is it just this?

shared interface Summable<Other> of Other ... {
    shared formal static Other additiveIdentity;
    ...
}
shared native final class Integer(Integer integer) ... {
    shared actual static Integer additiveIdentity => 0;
    ...
}