Cannot type solve lambdas with parameter inside inner block statements when solving method calls

[tnterdan]

Expression context: .filter(parsed -> parsed != null && parsed.isSolved()) (parsed should be of type SymbolReference<MethodDeclaration>) Expression: parsed.isSolved() Stack trace:

java.lang.UnsupportedOperationException: TypeVariable {java.util.function.Predicate.T} is not a Reference Type
        at com.github.javaparser.symbolsolver.model.typesystem.Type.asReferenceType(Type.java:110)
        at com.github.javaparser.symbolsolver.javaparsermodel.contexts.MethodCallExprContext.solveMethod(MethodCallExprContext.java:171)
        at com.github.javaparser.symbolsolver.javaparsermodel.JavaParserFacade.solve(JavaParserFacade.java:238)
        at com.github.javaparser.symbolsolver.javaparsermodel.JavaParserFacade.solve(JavaParserFacade.java:144)

Refers to these lines in solveMethod():

else if (typeOfScope.isConstraint()){
                return MethodResolutionLogic.solveMethodInType(typeOfScope.asConstraintType().getBound().asReferenceType().getTypeDeclaration(), name, argumentsTypes, typeSolver);

[tnterdan]

Looking into the issue more, it seems like it's more specifically happening with try/catch blocks inside of lambda expressions.

[xdrop]

@tnterdan Can you share a small code sample (the whole stream/lambda) to try and investigate this further?

[tnterdan]

I'm still figuring out exactly what the root cause is, but here is some sample code that seems to reproduce it reliably (based on my own code).

I dug around in JSS code and found that if a lambda statement encapsulated in a BlockStmt, it assumes the last line is the return value and thus determines the type. So I tried changing it to find all of the ReturnStmt nodes and then find the type from all of them to see if that would at least give me the right return type on a statement by statement basis: List<ReturnStmt> returnStatements = blockStmt.getChildNodesByType(ReturnStmt.class);

However, now what seems to be happening is it is unable to solve the type of statement in new JavaParserFacade().solve(statement). I'm guessing maybe it's missing the context of what the statement variable represents when it tries to solve it.

public class JavaTest {
    class MethodDeclaration {
        public <T> List<T> getNodesByType(Class<T> clazz) {
            return new ArrayList<T>();
        }
    }
    class JavaParserFacade {
        public Solved solve(MethodDeclaration method) {
            return new Solved();
        }
    }
    class Solved {
        public boolean isSolved() {
            return true;
        }
    }
    private List<String> foo(MethodDeclaration methodDecl) {
        return methodDecl
                .getNodesByType(MethodDeclaration.class)
                .stream()
                .map(statement -> {
                    try {
                        return new JavaParserFacade().solve(statement);
                    } catch (Throwable e) {
                        return null;
                    }
                })
                .filter(parsed -> parsed != null && parsed.isSolved());
    }
}

Stack trace below. I've been trying to fix it myself, so the line numbers are likely to be inaccurate:

java.lang.RuntimeException: Issue calculating the type of the scope of MethodCallExprContext{wrapped=methodDecl.getNodesByType(MethodDeclaration.class).stream().map(statement -> {
    try {
        return new JavaParserFacade().solve(statement);
    } catch (Throwable e) {
        return null;
    }
}).filter(parsed -> parsed != null && parsed.isSolved())}

    at com.github.javaparser.symbolsolver.javaparsermodel.contexts.MethodCallExprContext.solveMethod(MethodCallExprContext.java:151)
    at com.github.javaparser.symbolsolver.javaparsermodel.JavaParserFacade.solve(JavaParserFacade.java:261)
    at com.github.javaparser.symbolsolver.javaparsermodel.JavaParserFacade.solve(JavaParserFacade.java:145)
    at com.github.javaparser.symbolsolver.Issue200.lambdaTypeParameterScopeIssue(Issue200.java:30)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke(Method.java:498)
    at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:50)
    at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
    at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:47)
    at org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
    at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:325)
    at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:78)
    at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:57)
    at org.junit.runners.ParentRunner$3.run(ParentRunner.java:290)
    at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:71)
    at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:288)
    at org.junit.runners.ParentRunner.access$000(ParentRunner.java:58)
    at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:268)
    at org.junit.runners.ParentRunner.run(ParentRunner.java:363)
    at org.junit.runner.JUnitCore.run(JUnitCore.java:137)
    at com.intellij.junit4.JUnit4IdeaTestRunner.startRunnerWithArgs(JUnit4IdeaTestRunner.java:68)
    at com.intellij.rt.execution.junit.IdeaTestRunner$Repeater.startRunnerWithArgs(IdeaTestRunner.java:51)
    at com.intellij.rt.execution.junit.JUnitStarter.prepareStreamsAndStart(JUnitStarter.java:237)
    at com.intellij.rt.execution.junit.JUnitStarter.main(JUnitStarter.java:70)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke(Method.java:498)
    at com.intellij.rt.execution.application.AppMain.main(AppMain.java:147)
Caused by: java.lang.RuntimeException: Error calculating the type of parameter statement of method call new JavaParserFacade().solve(statement)
    at com.github.javaparser.symbolsolver.javaparsermodel.JavaParserFacade.solveMethodAsUsage(JavaParserFacade.java:500)
    at com.github.javaparser.symbolsolver.javaparsermodel.TypeExtractor.visit(TypeExtractor.java:263)
    at com.github.javaparser.symbolsolver.javaparsermodel.TypeExtractor.visit(TypeExtractor.java:45)
    at com.github.javaparser.ast.expr.MethodCallExpr.accept(MethodCallExpr.java:86)
    at com.github.javaparser.symbolsolver.javaparsermodel.JavaParserFacade.getTypeConcrete(JavaParserFacade.java:395)
    at com.github.javaparser.symbolsolver.javaparsermodel.JavaParserFacade.getType(JavaParserFacade.java:286)
    at com.github.javaparser.symbolsolver.javaparsermodel.JavaParserFacade.getType(JavaParserFacade.java:280)
    at com.github.javaparser.symbolsolver.javaparsermodel.TypeExtractor.visit(TypeExtractor.java:426)
    at com.github.javaparser.symbolsolver.javaparsermodel.TypeExtractor.visit(TypeExtractor.java:45)
    at com.github.javaparser.ast.expr.LambdaExpr.accept(LambdaExpr.java:117)
    at com.github.javaparser.symbolsolver.javaparsermodel.JavaParserFacade.getTypeConcrete(JavaParserFacade.java:395)
    at com.github.javaparser.symbolsolver.javaparsermodel.JavaParserFacade.getType(JavaParserFacade.java:286)
    at com.github.javaparser.symbolsolver.javaparsermodel.JavaParserFacade.getType(JavaParserFacade.java:295)
    at com.github.javaparser.symbolsolver.javaparsermodel.JavaParserFacade.getType(JavaParserFacade.java:280)
    at com.github.javaparser.symbolsolver.javaparsermodel.contexts.MethodCallExprContext.solveMethod(MethodCallExprContext.java:149)
    ... 30 more
Caused by: java.lang.UnsupportedOperationException: com.github.javaparser.ast.type.UnknownType
    at com.github.javaparser.symbolsolver.javaparsermodel.JavaParserFacade.convertToUsage(JavaParserFacade.java:478)
    at com.github.javaparser.symbolsolver.javaparsermodel.JavaParserFacade.convert(JavaParserFacade.java:488)
    at com.github.javaparser.symbolsolver.javaparsermodel.JavaParserFacade.convert(JavaParserFacade.java:484)
    at com.github.javaparser.symbolsolver.javaparsermodel.declarations.JavaParserParameterDeclaration.getType(JavaParserParameterDeclaration.java:66)
    at com.github.javaparser.symbolsolver.model.resolution.Value.from(Value.java:40)
    at com.github.javaparser.symbolsolver.core.resolution.Context.solveSymbolAsValue(Context.java:61)
    at com.github.javaparser.symbolsolver.javaparsermodel.contexts.StatementContext.solveSymbolAsValue(StatementContext.java:95)
    at com.github.javaparser.symbolsolver.javaparsermodel.contexts.StatementContext.solveSymbolAsValue(StatementContext.java:117)
    at com.github.javaparser.symbolsolver.javaparsermodel.contexts.MethodCallExprContext.solveSymbolAsValue(MethodCallExprContext.java:129)
    at com.github.javaparser.symbolsolver.resolution.SymbolSolver.solveSymbolAsValue(SymbolSolver.java:66)
    at com.github.javaparser.symbolsolver.resolution.SymbolSolver.solveSymbolAsValue(SymbolSolver.java:71)
    at com.github.javaparser.symbolsolver.javaparsermodel.TypeExtractor.visit(TypeExtractor.java:273)
    at com.github.javaparser.symbolsolver.javaparsermodel.TypeExtractor.visit(TypeExtractor.java:45)
    at com.github.javaparser.ast.expr.NameExpr.accept(NameExpr.java:65)
    at com.github.javaparser.symbolsolver.javaparsermodel.JavaParserFacade.getTypeConcrete(JavaParserFacade.java:395)
    at com.github.javaparser.symbolsolver.javaparsermodel.JavaParserFacade.getType(JavaParserFacade.java:315)
    at com.github.javaparser.symbolsolver.javaparsermodel.JavaParserFacade.solveMethodAsUsage(JavaParserFacade.java:498)
    ... 44 more

[xdrop]

Might have a fix for this with 7829f71ab529be59fabd3b78325b624de82f1815

Will test and let you know

[tnterdan]

Awesome, thanks so much for working on it! It fixes the issue for my case 👍

I took a look at your fix and found an additional test case you might want to try out:

public class JavaTest {
    class MethodDeclaration {
        public <T> List<T> getNodesByType(Class<T> clazz) {
            return new ArrayList<T>();
        }
    }
    class JavaParserFacade {
        public Solved solve(MethodDeclaration method) {
            return new Solved();
        }
        public SuperSolved superSolve(MethodDeclaration method) {
            return new SuperSolved();
        }
    }
    class Solved extends SuperSolved {
    }
    class SuperSolved {
        public boolean isSolved() {
            return true;
        }
    }
    private List<String> foo(MethodDeclaration methodDecl) {
        return methodDecl
                .getNodesByType(MethodDeclaration.class)
                .stream()
                .map(statement -> {
                    try {
                        if (true) {
                            return new JavaParserFacade().superSolve(statement);
                        } else {
                            return new JavaParserFacade().solve(statement);
                        }
                    } catch (Throwable e) {
                        return null;
                    }
                })
                .filter(parsed -> parsed != null && parsed.isSolved());
    }
}

Testing it out, depending on whether the test code code has the SuperSolve or the Solve appear first, the stream becomes the type of whichever is first. But it should really always be SuperSolve (the nearest common super class).

Also Java does not have an implicit return inside a block (only for lambdas without a block), so if you can't find any return statements, then there is no return type.

Some examples: https://docs.oracle.com/javase/specs/jls/se8/html/jls-15.html#jls-15.27 () -> { System.gc(); } // No parameters, void block body

[xdrop]

@tnterdan Thanks for the heads up on implicit return I've removed the code that handles that.

As for your test case I don't think we have yet a way of inferring the most specific shared supertype, and there are cases like:

interface A {}
interface B{}
interface C1 extends A,B{}
interface C2 extends A,B{}

return items.stream().map(x -> {
    if(cond1){ return (C1) null; }
    else     { return (C2) null; }
}.findFirst().get();

The type could then be either A or B. The way that Java handles this is it imposes constraints based on every usage of the type to produce constraint formulas and reduce them to the most appropriate one (e.g if the method was listed to return A it would infer that the type is A). Our approach to type solving is a bit different and it would be quite a major undertaking to work around that.

[tnterdan]

Looks like a tricky problem :) Thanks for the response! And thanks for fixing for the simple case :+1: