search

amino-os / Amino.Run

Amino Distributed OS - Runtime Manager
Apache License 2.0
29 stars 12 forks source link

hanksTodoRuby:runrubyapp fails with 'Message not supported: INVOKE' #424

Closed quinton-hoole closed 5 years ago

quinton-hoole commented 5 years ago

Task :examples:hanksTodoRuby:runrubyapp /Users/quinton/projects/Amino/DCAP-Sapphire/sapphire/examples/hanksTodoRuby/src/main/ruby/sapphire/appexamples/hankstodo/hanks_todo_main.rb:34:in main': Message not supported: INVOKE (UnsupportedMessageException) (RuntimeError) from com.oracle.truffle.api.interop.UnsupportedMessageException.raise(UnsupportedMessageException.java:68) from com.oracle.truffle.api.interop.java.JavaObjectMessageResolution$InvokeNode.access(JavaObjectMessageResolution.java:91) from com.oracle.truffle.api.interop.java.JavaObjectMessageResolutionForeign$InvokeSubNode.accessWithTarget(JavaObjectMessageResolutionForeign.java:256) from com.oracle.truffle.api.interop.java.JavaObjectMessageResolutionForeignFactory$InvokeSubNodeGen.executeAndSpecialize(JavaObjectMessageResolutionForeignFactory.java:270) from com.oracle.truffle.api.interop.java.JavaObjectMessageResolutionForeignFactory$InvokeSubNodeGen.executeWithTarget(JavaObjectMessageResolutionForeignFactory.java:252) from com.oracle.truffle.api.interop.java.JavaObjectMessageResolutionForeign$InvokeSubNode$INVOKERootNode.execute(JavaObjectMessageResolutionForeign.java:279) from org.graalvm.compiler.truffle.runtime.OptimizedCallTarget.callProxy(OptimizedCallTarget.java:269) from org.graalvm.compiler.truffle.runtime.OptimizedCallTarget.callRoot(OptimizedCallTarget.java:258) from org.graalvm.compiler.truffle.runtime.OptimizedCallTarget.callBoundary(OptimizedCallTarget.java:248) from org.graalvm.compiler.truffle.runtime.OptimizedCallTarget.doInvoke(OptimizedCallTarget.java:230) Translated to internal error from /Users/quinton/projects/Amino/DCAP-Sapphire/sapphire/examples/hanksTodoRuby/src/main/ruby/sapphire/appexamples/hankstodo/hanks_todo_main.rb:40:in

' got the object Hanks add First task OK!

quinton-hoole commented 5 years ago

I'm on GraalVM RC5:

$ java -version openjdk version "1.8.0_172" OpenJDK Runtime Environment (build 1.8.0_172-20180626105433.graaluser.jdk8u-src-tar-g-b11) GraalVM 1.0.0-rc5 (build 25.71-b01-internal-jvmci-0.46, mixed mode)

quinton-hoole commented 5 years ago

I rebuilt polyglot, and now I get a different error:

$ /Users/quinton/Downloads/graalvm-ce-1.0.0-rc5/Contents/Home/jre/bin/gu rebuild-images polyglot
Building polyglot...
   classlist:   9,966.80 ms
       (cap):   1,714.80 ms
       setup:   4,988.95 ms
  (typeflow):  76,736.26 ms
   (objects):  67,013.98 ms
  (features):  10,446.60 ms
    analysis: 159,044.21 ms
15658 method(s) included for runtime compilation
    universe:  17,920.37 ms
     (parse):   7,485.03 ms
    (inline):  18,693.53 ms
   (compile):  36,605.65 ms
     compile:  68,503.55 ms
       image:  23,011.34 ms
       write: 472,014.00 ms
     [total]: 755,686.55 ms
Quintons-MacBook-Pro:sapphire quinton$ ./gradlew examples:hanksTodoRuby:runRubyApp

> Configure project :
Gradle now uses separate output directories for each JVM language, but this build assumes a single directory for all classes from a source set. This behaviour has been deprecated and is scheduled to be removed in Gradle 5.0
        at build_76gueay4s22lcw2edo3d1wzdy$_run_closure2$_closure8.doCall(/Users/quinton/projects/Amino/DCAP-Sapphire/sapphire/build.gradle:113)

> Task :examples:hanksTodoRuby:runrubyapp
/Users/quinton/projects/Amino/DCAP-Sapphire/sapphire/examples/hanksTodoRuby/src/main/ruby/sapphire/appexamples/hankstodo/hanks_todo_main.rb:32:in `main': java.lang.NullPointerException (HostException) (RuntimeError)
Caused by:
java.lang.NullPointerException (RuntimeException)
        from sapphire.appexamples.hankstodo.stubs.TodoListManager_Stub.getTodos(TodoListManager_Stub.java:211)
        from com.oracle.truffle.api.interop.java.SingleMethodDesc$MHBase.invokeHandle(SingleMethodDesc.java:234)
        from com.oracle.truffle.api.interop.java.SingleMethodDesc$MHBase.invoke(SingleMethodDesc.java:226)
        from com.oracle.truffle.api.interop.java.ExecuteMethodNode.doInvoke(ExecuteMethodNode.java:706)
        from com.oracle.truffle.api.interop.java.ExecuteMethodNode.doVarArgs(ExecuteMethodNode.java:136)
        from com.oracle.truffle.api.interop.java.ExecuteMethodNodeGen.executeAndSpecialize(ExecuteMethodNodeGen.java:146)
        from com.oracle.truffle.api.interop.java.ExecuteMethodNodeGen.executeImpl(ExecuteMethodNodeGen.java:79)
        from com.oracle.truffle.api.interop.java.ExecuteMethodNode.execute(ExecuteMethodNode.java:67)
        from com.oracle.truffle.api.interop.java.JavaObjectMessageResolution$InvokeNode.access(JavaObjectMessageResolution.java:100)
        from com.oracle.truffle.api.interop.java.JavaObjectMessageResolutionForeign$InvokeSubNode.accessWithTarget(JavaObjectMessageResolutionForeign.java:256)
Caused by:
<no message> (NullPointerException)
        from sapphire.appexamples.hankstodo.stubs.TodoListManager_Stub.getTodos(TodoListManager_Stub.java:201)
        from com.oracle.truffle.api.interop.java.SingleMethodDesc$MHBase.invokeHandle(SingleMethodDesc.java:234)
        from com.oracle.truffle.api.interop.java.SingleMethodDesc$MHBase.invoke(SingleMethodDesc.java:226)
        from com.oracle.truffle.api.interop.java.ExecuteMethodNode.doInvoke(ExecuteMethodNode.java:706)
        from com.oracle.truffle.api.interop.java.ExecuteMethodNode.doVarArgs(ExecuteMethodNode.java:136)
        from com.oracle.truffle.api.interop.java.ExecuteMethodNodeGen.executeAndSpecialize(ExecuteMethodNodeGen.java:146)
        from com.oracle.truffle.api.interop.java.ExecuteMethodNodeGen.executeImpl(ExecuteMethodNodeGen.java:79)
        from com.oracle.truffle.api.interop.java.ExecuteMethodNode.execute(ExecuteMethodNode.java:67)
        from com.oracle.truffle.api.interop.java.JavaObjectMessageResolution$InvokeNode.access(JavaObjectMessageResolution.java:100)
        from com.oracle.truffle.api.interop.java.JavaObjectMessageResolutionForeign$InvokeSubNode.accessWithTarget(JavaObjectMessageResolutionForeign.java:256)
Translated to internal error
        from /Users/quinton/projects/Amino/DCAP-Sapphire/sapphire/examples/hanksTodoRuby/src/main/ruby/sapphire/appexamples/hankstodo/hanks_todo_main.rb:40:in `<main>'
got the object Hanks
quinton-hoole commented 5 years ago

Never mind. I restarted all my processes after the build, and that seems to sort it out.

> Task :examples:hanksTodoRuby:runrubyapp
got the object Hanks
add First task OK!
add Second task Failed !!!!
add Third task Failed !!!!
Current Task List:
First todo
Update Second task status OK!
Current Task List:
First todo

BUILD SUCCESSFUL in 26s
quinton-hoole commented 5 years ago

@haibinmichaeljie saw this again, so te-opening

MichaelHaibinXie commented 5 years ago
screen shot 2018-11-13 at 10 36 29 am screen shot 2018-11-13 at 10 36 42 am
MichaelHaibinXie commented 5 years ago

@quinton-hoole I noticed in your last screenshot, it also failed as you can see "add second task failed" and "add third task failed"

MichaelHaibinXie commented 5 years ago

I am trying to debug it however the exception keeps changing, there are several different failures, and if you run several times you may see different exceptions...

MichaelHaibinXie commented 5 years ago

looks like recent regression.

sungwook-moon commented 5 years ago

I noticed the failure before creating PR #423. You can see the exception detail in the PR as well. Therefore, I suggest looking at the PRs before that time.

MichaelHaibinXie commented 5 years ago

I think sapphire object method call fails occasionally, we have not implemented and/or tested how sapphire object failure/exceptions get propagated to the client app. that should be a separate task to do.

MichaelHaibinXie commented 5 years ago

The good thing is kvstorejs works consistently.

MichaelHaibinXie commented 5 years ago

@AmitRoushan @Sungwook Moon @Quinton Hoole This is caused by DHT policy. when shards is 2, newTodoList, addTodo, getTodo are messed up, how do we distribute the request to shard? is newTodoList only applied on one shard?. Please propose a fix.

sungwook-moon commented 5 years ago

@MichaelHaibinXie Could you be more specific? Does it add 2+ servers (or nodes) to the group in DHT for example in case Shared number was supposed to be only 2? Also, could you try after syncing with the recent code as KVstore demo using DHT+LoadBalancedMasterSlave was fixed?

MichaelHaibinXie commented 5 years ago

In the HanksTodoJS.yaml. it uses this policy. my code was up to date. KeyValueStore always works, when client makes newTodoList, which shard gets the request? how does it guarantee the same shard gets addTodo requests. If you change the numOfShards to 1 in below config the issue goes away.

MichaelHaibinXie commented 5 years ago

@Sungwook Moon This is caused by this code, I commented it out and it worked all well. I am assigning this bug to you. In DHTPolicy.java // TODO (Sungwook, 2018-10-2) Passing processedPolicies may not be necessary as // they are already available. //SapphireServerPolicy replica = // dhtServer.sapphire_replicate(server.getProcessedPolicies(), region);

sungwook-moon commented 5 years ago

Update: I did an investigation and found HanksTodoRuby started firing this exception from PR #414 : "fixes #395: make user defined data structure work as parameter for graal method invocation"

This PR touches files such as sapphire/examples/hanksTodoRuby/src/main/java/sapphire/appexamples/hankstodo/Client.java which I am not familiar with at all.

I can look into it but think someone familiar with above codebase may be better suited.

sungwook-moon commented 5 years ago

@AmitRoushan Any update on this?

AmitRoushan commented 5 years ago

@sungwook-moon raised PR with fix. Please have a look.