Java 8: support auto-SMI of our Callable types

[CeylonMigrationBot]

[@FroMage] In theory we should be able to pretend that any method in Java which accepts an SMI (Single-Method Interface) accepts a union of the SMI and its corresponding Ceylon Callable, and do the magic auto-conversion in our backend, when provided with a Callable.

Similarly we should be able to pretend that any method in Java which returns an SMI returns an intersection of the SMI and its corresponding Ceylon Callable, and do the magic auto-conversion in our backend.

I'm not 100% convinced we can do this due to the usual issues with types and bounds and type-arguments, but we can try and investigate.

I highly doubt we'll have time to do this for 1.1, so marking 1.2

[Migrated from ceylon/ceylon-compiler#1617]

[FroMage]

And the fix is to finally bite the bullet and get rid of the special cases in transformeExpression yay!

[fwgreen]

CompletableFuture.supplyAsync(() => repo.listAll(), mes.execute)

error: Ceylon backend error: incompatible types: <anonymous AbstractCallable<Object>> cannot be converted to Executor

I should point out that the messy way...

CompletableFuture.supplyAsync(object satisfies Supplier<List<Person>> { get() => repo.listAll(); }, mes)

continues to work, so we're not dead in the water.

As an aside, Kotlin currently has the same limitation, but uses a shorthand along the lines of https://github.com/ceylon/ceylon/issues/5739 to ease the pain.

CompletableFuture.supplyAsync(Supplier { repo.listAll() }, mes )

[FroMage]

Yeah I didn't have time to push my fix yet

[FroMage]

Damn, I forgot coercions to invocations :(

[FroMage]

@fwgreen it should work now.

[FroMage]

OK invocations work. And the metamodel is not affected because the runtime metamodel will not report functional interfaces: the overloads don't exist at runtime.

[FroMage]

Now the IDEs have to implement the model loader methods for this to work in the IDEs.

[bjansen]

What should happen when a class defines both

Future<?> executeOnPooledThread(@NotNull Runnable action);

and

<T> Future<T> executeOnPooledThread(@NotNull Callable<T> action);

? If I use the following argument:

void submitChangesTask() { }

I get this error:

[ceylon-compile] /Users/bastien/Dev/ceylon/ceylon-ide-intellij/plugin-ceylon-code/source/org/intellij/plugins/ceylon/ide/ceylonCode/model/CeylonModelManager.ceylon:272: error: argument must be assignable to parameter 'arg0' of 'executeOnPooledThread' in 'Application': 'Anything()' is not assignable to 'Object()'
[ceylon-compile]             => submitChangesFuture = application.executeOnPooledThread(submitChangesTask);
[ceylon-compile]                                                                        ^

I can't disambiguate using of:

[ceylon-compile] /Users/bastien/Dev/ceylon/ceylon-ide-intellij/plugin-ceylon-code/source/org/intellij/plugins/ceylon/ide/ceylonCode/model/CeylonModelManager.ceylon:272: error: specified type does not cover the cases of the operand expression: 'Runnable' does not cover 'Anything()'
[ceylon-compile]             => submitChangesFuture = application.executeOnPooledThread(submitChangesTask of Runnable);
[ceylon-compile]                                                                        ^

[bjansen]

Oh, it looks like Object submitChangesTask() => null; is accepted. My model loader accepts both versions, so I'm not sure if I implemented it correctly or not?

[FroMage]

To be honest that's a use-case we didn't think about: overloading where there are two exactly similar SAM types. I don't think the overloading resolution can pick either one. WDYT @gavinking ?

[gavinking]

I think it's very easy to use object Runnable { run=submitChangesTask; } in such cases. I don't think we should be worried about this one.

What API is it anyway, @bjansen?

[bjansen]

IntelliJ's Application

[bjansen]

I agree it's easy to use an object instead, but perhaps the TC should throw an error saying that this isn't possible on overloads?

[fwgreen]

It's great that this now works

shared default void list(suspended AsyncResponse response) {
        CompletableFuture.supplyAsync(repo.listAll)
                         .thenApply(generic)
                         .thenAccept((list) => response.resume(list));
    }

    GenericEntity<List<Person>> generic(List<Person> list) => object extends GenericEntity<List<Person>>(list) {};

Feeling lucky (greedy), I tried using a Java method reference

...thenAccept(response.resume);

which triggers

error: ambiguous callable reference to overloaded method or class: 'resume' is overloaded
                         .thenAccept(response.resume);

[gavinking]

Feeling lucky (greedy), I tried using a Java method reference

That's correct. We've never supported method refs to overloaded Java methods. For pretty good reasons.

[jvasileff]

Is it intentional that optional return types aren't allowed?

I image needing to return null is much less common than wanting to have non-optional parameter types (which is allowed). But still, it may be useful at times.

[gavinking]

Is it intentional that optional return types aren't allowed?

@jvasileff Example?

@FroMage is this perhaps just a matter of setting uncheckedNulls correctly in all the right places?

[FroMage]

That's still a TODO, I haven't tested this at all.

[jvasileff]

@gavinking anything, really. But for testing, something like:

import java.util.concurrent { CompletableFuture }

void foo(CompletableFuture<String> future) {
    CompletableFuture<String> newFuture
        = future.thenApply<String>((String? s) => s);
}

[FroMage]

is this perhaps just a matter of setting uncheckedNulls correctly in all the right places?

Nope, this is on TypedDeclaration and not on Type (and we don't want it on Callable).

@gavinking not sure how best to proceed. I can fudge with the signature of the Callable but will it be enough?

[gavinking]

@FroMage I found a bug.

The parameter of Stream.map() gets the function type Nothing(String)?. That not right. It should be R(String)?.

[gavinking]

Also, I don't think we actually need to mark these callable parameter types optional. If for some dumb reason you want to pass null, which seems to me pretty unlikely, you can use the other overload, DYT?

[gavinking]

@gavinking not sure how best to proceed. I can fudge with the signature of the Callable but will it be enough?

I think probably the best thing we can do is arrange for the typechecker to set uncheckedNulls on an anonymous function when it gets passed as a parameter to a Java method.

Is isJava() set to true on the generated Value parameter? If not could you do me a favor and set it to true, please?

[FroMage]

The parameter of Stream.map() gets the function type Nothing(String)?. That not right. It should be R(String)?.

This is due to the fudging I just did or is it older?

[gavinking]

This is due to the fudging I just did or is it older?

Not sure. Probably older.

[FroMage]

If for some dumb reason you want to pass null, which seems to me pretty unlikely, you can use the other overload, DYT?

Well, not sure, depends on how common that is. I haven't used the stream API enough to judge.

[gavinking]

Well, not sure, depends on how common that is.

Let's review all the method of Stream, and see if any of them accept null.

[FroMage]

Well, it's not the methods of Stream, it's the functional types that accept or return null.

[FroMage]

Is isJava() set to true on the generated Value parameter? If not could you do me a favor and set it to true, please?

Done. Though you could have trusted the method.

[gavinking]

I think probably the best thing we can do is arrange for the typechecker to set uncheckedNulls on an anonymous function when it gets passed as a parameter to a Java method.

Actually this won't work for multiple reasons:

1. A Java SAM could have a return type like int or float.
2. Anonymous functions don't have a declared return type, so the inferred return type will always be optional.

So that's a dead end, it seems to me.

[gavinking]

Well, it's not the methods of Stream, it's the functional types that accept or return null.

Nono, I'm talking about how Stream.map() accepts R(T)? instead of R(T). That's nothing to do with the functional interface type, it depends on what map() itself accepts.

[FroMage]

I can't reproduce your issue with Stream.map, it seems fine here:

|  |  |  |  |  |  |  +  [QualifiedMemberExpression] (113:15-114:15) : Stream<Integer>(Integer(Integer)?) : Stream<Integer>.map<Integer> : function Stream<T>.map<R>(R(T)? arg0) => Stream<R>

[FroMage]

Nono, I'm talking about how Stream.map() accepts R(T)? instead of R(T)

But they do already. That's not related to what @jvasileff was asking for. He was asking about being able to pass functions that take and return optional parameters.

[FroMage]

Well, the model loader certainly has the information as to whether the SAM function type can accept null params or return null values. It's just a matter of passing that info the typechecker.

[gavinking]

But they do already.

Right, but I don't want them to.

That's not related to what @jvasileff was asking for.

I know I know, this is something different I'm asking for.

[FroMage]

Would it help if we generated types such as String?(Integer primitive, Integer? notPrimitive)?

[FroMage]

I know I know, this is something different I'm asking for.

If you're asking me you'll have to be clearer then ;) I've no idea.

[FroMage]

Why don't you want Stream functions to accept null parameters? What's bad about that?

[gavinking]

Would it help if we generated types such as String?(Integer primitive, Integer? notPrimitive)?

Yes, that's the right thing to do, I think. Return types of callable parameters are contravariant like the type of a parameter value. We usually generate ? in that case. It's in covariant locations that we use uncheckedNulls.

[gavinking]

Why don't you want Stream functions to accept null parameters? What's bad about that?

I'm scared it might mess with:

• type arg inference
• overload resolution

[gavinking]

It's in covariant locations that we use uncheckedNulls.

For example, to be completely consistent, we should produce the type String?(Integer primitive, Integer notPrimitive) in your example. And notPrimitive should have uncheckedNulls marked to true. But I don't think the typechecker would know what to do with that right now, so String?(Integer primitive, Integer? notPrimitive) is good enough for now.

[FroMage]

Yes, that's the right thing to do, I think. Return types of callable parameters are contravariant like the type of a parameter value. We usually generate ? in that case. It's covariant locations that we use uncheckedNulls..

Well, we're left with the problem of parameters. If I do that you can't pass a function(Integer prim, Foo notPrim) to a String?(Integer prim, Foo? notPrim), can you?

[gavinking]

Well, we're left with the problem of parameters. If I do that you can't pass a function(Integer prim, Foo notPrim) to a String?(Integer prim, Foo? notPrim), can you?

Correct. See my previous comment.

[FroMage]

And notPrimitive should have uncheckedNulls marked to true. But I don't think the typechecker would know what to do with that right now

Well, there's no place to mark it. It's not a Function with parameters: it's a Callable type. That can't be marked in any way for its parameters.

[gavinking]

But I don't think the typechecker would know what to do with that right now, so String?(Integer primitive, Integer? notPrimitive) is good enough for now.

Oh wait; screw that, that would mean the parameter of map() would get the type R?(T?) which would drive everyone nuts! No no, it has to be R(T) for sure.

[gavinking]

Well, there's no place to mark it. It's not a Function with parameters: it's a Callable type.

Just make it String?(Integer primitive, Integer notPrimitive) for now. We'll deal with null in the next release by generating a functional parameter instead of a value parameter.

[FroMage]

Well, I can create a functional parameter too, if that's easier?

[FroMage]

At least, if it doesn't impact the rest of the backend work I did…

[gavinking]

Well, I can create a functional parameter too, if that's easier?

I don't think it will help much right now, without some additional work in the typechecker.

[gavinking]

I can't reproduce your issue with Stream.map, it seems fine here:

My bad, it's an intellij-specific bug. Works fine in the CLI.