search

scipr-lab / dizk

Java library for distributed zero knowledge proof systems
Other
232 stars 66 forks source link

Test Error #12

Closed SilverPoker closed 4 years ago

SilverPoker commented 4 years ago

I followed the four steps and find errors when running mvn test The I use mvn text -x for printing out logs, the error log is below:

Tests run: 71, Failures: 0, Errors: 10, Skipped: 0

[INFO] ------------------------------------------------------------------------
[INFO] BUILD FAILURE
[INFO] ------------------------------------------------------------------------
[INFO] Total time:  44.968 s
[INFO] Finished at: 2020-02-21T21:40:08+08:00
[INFO] ------------------------------------------------------------------------
[ERROR] Failed to execute goal org.apache.maven.plugins:maven-surefire-plugin:2.12.4:test (default-test) on project dizk: There are test failures.
[ERROR] 
[ERROR] Please refer to /home/silverpoker/github/dizk/target/surefire-reports for the individual test results.
[ERROR] -> [Help 1]
org.apache.maven.lifecycle.LifecycleExecutionException: Failed to execute goal org.apache.maven.plugins:maven-surefire-plugin:2.12.4:test (default-test) on project dizk: There are test failures.

Please refer to /home/silverpoker/github/dizk/target/surefire-reports for the individual test results.
    at org.apache.maven.lifecycle.internal.MojoExecutor.execute (MojoExecutor.java:215)
    at org.apache.maven.lifecycle.internal.MojoExecutor.execute (MojoExecutor.java:156)
    at org.apache.maven.lifecycle.internal.MojoExecutor.execute (MojoExecutor.java:148)
    at org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject (LifecycleModuleBuilder.java:117)
    at org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject (LifecycleModuleBuilder.java:81)
    at org.apache.maven.lifecycle.internal.builder.singlethreaded.SingleThreadedBuilder.build (SingleThreadedBuilder.java:56)
    at org.apache.maven.lifecycle.internal.LifecycleStarter.execute (LifecycleStarter.java:128)
    at org.apache.maven.DefaultMaven.doExecute (DefaultMaven.java:305)
    at org.apache.maven.DefaultMaven.doExecute (DefaultMaven.java:192)
    at org.apache.maven.DefaultMaven.execute (DefaultMaven.java:105)
    at org.apache.maven.cli.MavenCli.execute (MavenCli.java:956)
    at org.apache.maven.cli.MavenCli.doMain (MavenCli.java:288)
    at org.apache.maven.cli.MavenCli.main (MavenCli.java:192)
    at jdk.internal.reflect.NativeMethodAccessorImpl.invoke0 (Native Method)
    at jdk.internal.reflect.NativeMethodAccessorImpl.invoke (NativeMethodAccessorImpl.java:62)
    at jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke (DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke (Method.java:566)
    at org.codehaus.plexus.classworlds.launcher.Launcher.launchEnhanced (Launcher.java:289)
    at org.codehaus.plexus.classworlds.launcher.Launcher.launch (Launcher.java:229)
    at org.codehaus.plexus.classworlds.launcher.Launcher.mainWithExitCode (Launcher.java:415)
    at org.codehaus.plexus.classworlds.launcher.Launcher.main (Launcher.java:356)
Caused by: org.apache.maven.plugin.MojoFailureException: There are test failures.

Please refer to /home/silverpoker/github/dizk/target/surefire-reports for the individual test results.
    at org.apache.maven.plugin.surefire.SurefireHelper.reportExecution (SurefireHelper.java:83)
    at org.apache.maven.plugin.surefire.SurefirePlugin.writeSummary (SurefirePlugin.java:176)
    at org.apache.maven.plugin.surefire.SurefirePlugin.handleSummary (SurefirePlugin.java:150)
    at org.apache.maven.plugin.surefire.AbstractSurefireMojo.executeAfterPreconditionsChecked (AbstractSurefireMojo.java:650)
    at org.apache.maven.plugin.surefire.AbstractSurefireMojo.execute (AbstractSurefireMojo.java:586)
    at org.apache.maven.plugin.DefaultBuildPluginManager.executeMojo (DefaultBuildPluginManager.java:137)
    at org.apache.maven.lifecycle.internal.MojoExecutor.execute (MojoExecutor.java:210)
    at org.apache.maven.lifecycle.internal.MojoExecutor.execute (MojoExecutor.java:156)
    at org.apache.maven.lifecycle.internal.MojoExecutor.execute (MojoExecutor.java:148)
    at org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject (LifecycleModuleBuilder.java:117)
    at org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject (LifecycleModuleBuilder.java:81)
    at org.apache.maven.lifecycle.internal.builder.singlethreaded.SingleThreadedBuilder.build (SingleThreadedBuilder.java:56)
    at org.apache.maven.lifecycle.internal.LifecycleStarter.execute (LifecycleStarter.java:128)
    at org.apache.maven.DefaultMaven.doExecute (DefaultMaven.java:305)
    at org.apache.maven.DefaultMaven.doExecute (DefaultMaven.java:192)
    at org.apache.maven.DefaultMaven.execute (DefaultMaven.java:105)
    at org.apache.maven.cli.MavenCli.execute (MavenCli.java:956)
    at org.apache.maven.cli.MavenCli.doMain (MavenCli.java:288)
    at org.apache.maven.cli.MavenCli.main (MavenCli.java:192)
    at jdk.internal.reflect.NativeMethodAccessorImpl.invoke0 (Native Method)
    at jdk.internal.reflect.NativeMethodAccessorImpl.invoke (NativeMethodAccessorImpl.java:62)
    at jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke (DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke (Method.java:566)
    at org.codehaus.plexus.classworlds.launcher.Launcher.launchEnhanced (Launcher.java:289)
    at org.codehaus.plexus.classworlds.launcher.Launcher.launch (Launcher.java:229)
    at org.codehaus.plexus.classworlds.launcher.Launcher.mainWithExitCode (Launcher.java:415)
    at org.codehaus.plexus.classworlds.launcher.Launcher.main (Launcher.java:356)
[ERROR] 
[ERROR] Re-run Maven using the -X switch to enable full debug logging.
[ERROR] 
[ERROR] For more information about the errors and possible solutions, please read the following articles:
[ERROR] [Help 1] http://cwiki.apache.org/confluence/display/MAVEN/MojoFailureException
SilverPoker commented 4 years ago

The detailed error information of one of the failed tests is shown below:

-------------------------------------------------------------------------------
Test set: common.DistributedNaiveEvaluationTest
-------------------------------------------------------------------------------
Tests run: 1, Failures: 0, Errors: 1, Skipped: 0, Time elapsed: 0.199 sec <<< FAILURE!
DistributedNaiveTest(common.DistributedNaiveEvaluationTest)  Time elapsed: 0.199 sec  <<< ERROR!
org.apache.spark.SparkException: Job aborted due to stage failure: Task 0 in stage 1.0 failed 1 times, most recent failure: Lost task 0.0 in stage 1.0 (TID 1, localhost): java.io.IOException: unexpected exception type
    at java.base/java.io.ObjectStreamClass.throwMiscException(ObjectStreamClass.java:1626)
    at java.base/java.io.ObjectStreamClass.invokeReadResolve(ObjectStreamClass.java:1256)
    at java.base/java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:2151)
    at java.base/java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1646)
    at java.base/java.io.ObjectInputStream.defaultReadFields(ObjectInputStream.java:2410)
    at java.base/java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:2304)
    at java.base/java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:2142)
    at java.base/java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1646)
    at java.base/java.io.ObjectInputStream.defaultReadFields(ObjectInputStream.java:2410)
    at java.base/java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:2304)
    at java.base/java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:2142)
    at java.base/java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1646)
    at java.base/java.io.ObjectInputStream.defaultReadFields(ObjectInputStream.java:2410)
    at java.base/java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:2304)
    at java.base/java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:2142)
    at java.base/java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1646)
    at java.base/java.io.ObjectInputStream.defaultReadFields(ObjectInputStream.java:2410)
    at java.base/java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:2304)
    at java.base/java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:2142)
    at java.base/java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1646)
    at java.base/java.io.ObjectInputStream.readObject(ObjectInputStream.java:464)
    at java.base/java.io.ObjectInputStream.readObject(ObjectInputStream.java:422)
    at org.apache.spark.serializer.JavaDeserializationStream.readObject(JavaSerializer.scala:75)
    at org.apache.spark.serializer.JavaSerializerInstance.deserialize(JavaSerializer.scala:114)
    at org.apache.spark.scheduler.ResultTask.runTask(ResultTask.scala:66)
    at org.apache.spark.scheduler.Task.run(Task.scala:85)
    at org.apache.spark.executor.Executor$TaskRunner.run(Executor.scala:274)
    at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
    at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
    at java.base/java.lang.Thread.run(Thread.java:834)
Caused by: java.lang.reflect.InvocationTargetException
    at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
    at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.base/java.lang.reflect.Method.invoke(Method.java:566)
    at java.base/java.lang.invoke.SerializedLambda.readResolve(SerializedLambda.java:237)
    at java.base/jdk.internal.reflect.GeneratedMethodAccessor11.invoke(Unknown Source)
    at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.base/java.lang.reflect.Method.invoke(Method.java:566)
    at java.base/java.io.ObjectStreamClass.invokeReadResolve(ObjectStreamClass.java:1250)
    ... 28 more
Caused by: java.lang.IllegalArgumentException: Invalid lambda deserialization
    at common.NaiveEvaluation.$deserializeLambda$(NaiveEvaluation.java:1)
    ... 37 more

Driver stacktrace:
    at org.apache.spark.scheduler.DAGScheduler.org$apache$spark$scheduler$DAGScheduler$$failJobAndIndependentStages(DAGScheduler.scala:1450)
    at org.apache.spark.scheduler.DAGScheduler$$anonfun$abortStage$1.apply(DAGScheduler.scala:1438)
    at org.apache.spark.scheduler.DAGScheduler$$anonfun$abortStage$1.apply(DAGScheduler.scala:1437)
    at scala.collection.mutable.ResizableArray$class.foreach(ResizableArray.scala:59)
    at scala.collection.mutable.ArrayBuffer.foreach(ArrayBuffer.scala:47)
    at org.apache.spark.scheduler.DAGScheduler.abortStage(DAGScheduler.scala:1437)
    at org.apache.spark.scheduler.DAGScheduler$$anonfun$handleTaskSetFailed$1.apply(DAGScheduler.scala:811)
    at org.apache.spark.scheduler.DAGScheduler$$anonfun$handleTaskSetFailed$1.apply(DAGScheduler.scala:811)
    at scala.Option.foreach(Option.scala:236)
    at org.apache.spark.scheduler.DAGScheduler.handleTaskSetFailed(DAGScheduler.scala:811)
    at org.apache.spark.scheduler.DAGSchedulerEventProcessLoop.doOnReceive(DAGScheduler.scala:1659)
    at org.apache.spark.scheduler.DAGSchedulerEventProcessLoop.onReceive(DAGScheduler.scala:1618)
    at org.apache.spark.scheduler.DAGSchedulerEventProcessLoop.onReceive(DAGScheduler.scala:1607)
    at org.apache.spark.util.EventLoop$$anon$1.run(EventLoop.scala:48)
    at org.apache.spark.scheduler.DAGScheduler.runJob(DAGScheduler.scala:632)
    at org.apache.spark.SparkContext.runJob(SparkContext.scala:1871)
    at org.apache.spark.SparkContext.runJob(SparkContext.scala:1934)
    at org.apache.spark.rdd.RDD$$anonfun$reduce$1.apply(RDD.scala:983)
    at org.apache.spark.rdd.RDDOperationScope$.withScope(RDDOperationScope.scala:151)
    at org.apache.spark.rdd.RDDOperationScope$.withScope(RDDOperationScope.scala:112)
    at org.apache.spark.rdd.RDD.withScope(RDD.scala:358)
    at org.apache.spark.rdd.RDD.reduce(RDD.scala:965)
    at org.apache.spark.api.java.JavaRDDLike$class.reduce(JavaRDDLike.scala:384)
    at org.apache.spark.api.java.AbstractJavaRDDLike.reduce(JavaRDDLike.scala:45)
    at common.NaiveEvaluation.parallelEvaluatePolynomial(NaiveEvaluation.java:46)
    at common.DistributedNaiveEvaluationTest.DistributedNaiveTest(DistributedNaiveEvaluationTest.java:53)
    at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
    at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.base/java.lang.reflect.Method.invoke(Method.java:566)
    at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:47)
    at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
    at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:44)
    at org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
    at org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:26)
    at org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:27)
    at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:271)
    at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:70)
    at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:50)
    at org.junit.runners.ParentRunner$3.run(ParentRunner.java:238)
    at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:63)
    at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:236)
    at org.junit.runners.ParentRunner.access$000(ParentRunner.java:53)
    at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:229)
    at org.junit.runners.ParentRunner.run(ParentRunner.java:309)
    at org.apache.maven.surefire.junit4.JUnit4Provider.execute(JUnit4Provider.java:252)
    at org.apache.maven.surefire.junit4.JUnit4Provider.executeTestSet(JUnit4Provider.java:141)
    at org.apache.maven.surefire.junit4.JUnit4Provider.invoke(JUnit4Provider.java:112)
    at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
    at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.base/java.lang.reflect.Method.invoke(Method.java:566)
    at org.apache.maven.surefire.util.ReflectionUtils.invokeMethodWithArray(ReflectionUtils.java:189)
    at org.apache.maven.surefire.booter.ProviderFactory$ProviderProxy.invoke(ProviderFactory.java:165)
    at org.apache.maven.surefire.booter.ProviderFactory.invokeProvider(ProviderFactory.java:85)
    at org.apache.maven.surefire.booter.ForkedBooter.runSuitesInProcess(ForkedBooter.java:115)
    at org.apache.maven.surefire.booter.ForkedBooter.main(ForkedBooter.java:75)
Caused by: java.io.IOException: unexpected exception type
    at java.base/java.io.ObjectStreamClass.throwMiscException(ObjectStreamClass.java:1626)
    at java.base/java.io.ObjectStreamClass.invokeReadResolve(ObjectStreamClass.java:1256)
    at java.base/java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:2151)
    at java.base/java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1646)
    at java.base/java.io.ObjectInputStream.defaultReadFields(ObjectInputStream.java:2410)
    at java.base/java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:2304)
    at java.base/java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:2142)
    at java.base/java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1646)
    at java.base/java.io.ObjectInputStream.defaultReadFields(ObjectInputStream.java:2410)
    at java.base/java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:2304)
    at java.base/java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:2142)
    at java.base/java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1646)
    at java.base/java.io.ObjectInputStream.defaultReadFields(ObjectInputStream.java:2410)
    at java.base/java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:2304)
    at java.base/java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:2142)
    at java.base/java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1646)
    at java.base/java.io.ObjectInputStream.defaultReadFields(ObjectInputStream.java:2410)
    at java.base/java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:2304)
    at java.base/java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:2142)
    at java.base/java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1646)
    at java.base/java.io.ObjectInputStream.readObject(ObjectInputStream.java:464)
    at java.base/java.io.ObjectInputStream.readObject(ObjectInputStream.java:422)
    at org.apache.spark.serializer.JavaDeserializationStream.readObject(JavaSerializer.scala:75)
    at org.apache.spark.serializer.JavaSerializerInstance.deserialize(JavaSerializer.scala:114)
    at org.apache.spark.scheduler.ResultTask.runTask(ResultTask.scala:66)
    at org.apache.spark.scheduler.Task.run(Task.scala:85)
    at org.apache.spark.executor.Executor$TaskRunner.run(Executor.scala:274)
    at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
    at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
    at java.base/java.lang.Thread.run(Thread.java:834)
Caused by: java.lang.reflect.InvocationTargetException
    at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
    at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.base/java.lang.reflect.Method.invoke(Method.java:566)
    at java.base/java.lang.invoke.SerializedLambda.readResolve(SerializedLambda.java:237)
    at java.base/jdk.internal.reflect.GeneratedMethodAccessor11.invoke(Unknown Source)
    at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.base/java.lang.reflect.Method.invoke(Method.java:566)
    at java.base/java.io.ObjectStreamClass.invokeReadResolve(ObjectStreamClass.java:1250)
    ... 28 more
Caused by: java.lang.IllegalArgumentException: Invalid lambda deserialization
    at common.NaiveEvaluation.$deserializeLambda$(NaiveEvaluation.java:1)
    ... 37 more
SilverPoker commented 4 years ago

I summarized the error information in the test as below: common.DistributedNaiveEvaluationTest

Caused by: java.lang.IllegalArgumentException: Invalid lambda deserialization
    at common.NaiveEvaluation.$deserializeLambda$(NaiveEvaluation.java:1)
    ... 37 more

reductions.r1cs.R1CSRelationRDDTest

Caused by: java.lang.IllegalArgumentException: Invalid lambda deserialization
    at relations.r1cs.R1CSRelationRDD.$deserializeLambda$(R1CSRelationRDD.java:1)
    ... 37 more

reductions.MatMulTest

Caused by: java.lang.IllegalArgumentException: Invalid lambda deserialization
    at relations.r1cs.R1CSRelationRDD.$deserializeLambda$(R1CSRelationRDD.java:1)
    ... 37 more

reductions.R1CSConstructionTest

Caused by: java.lang.IllegalArgumentException: Invalid lambda deserialization
    at relations.r1cs.R1CSRelationRDD.$deserializeLambda$(R1CSRelationRDD.java:1)
    ... 37 more

zk_proof_systems.zkSNARK.DistributdzkSNARKTest

Caused by: java.lang.IllegalArgumentException: Invalid lambda deserialization
    at relations.r1cs.R1CSRelationRDD.$deserializeLambda$(R1CSRelationRDD.java:1)
    ... 37 more
howardwu commented 4 years ago

@SilverPoker can you print out mvn -v? And are you compiling and running this in an IDE or terminal?

I wasn't able to reproduce the lambda deserialization errors on my machine. From an initial search online, it looks like Java lambdas are compiled differently depending on the compiler, which may have led to your issue (see here for example)

SilverPoker commented 4 years ago

Hi, thanks for your help! I tried using mvn clean and reinstalling the compiler, it works now.

By the way, I noticed that in the paper you have described two important algorithms but didn't really implement them. (1) For multiexp, the paper uses Pippenger’s algorithm but the code only has the implementation of Bos–Coster algorithm (2) For the setup phase, there is one way described in the paper to avoid the unbalanced sparse and dense tasks but the code only has the implementation of Strawman. There are still task unbalanced problems... Is my understanding correct? If there are some implementations of those two, could you please refer me to the file? Thanks!

howardwu commented 4 years ago

I'm glad the test error was resolved.

To answer your questions: (1) It indeed looks like the Pippenger's algorithm is missing from the library, very odd. I'll add an implementation in, thanks for catching that. (2) The sparse and dense tasks are processed in isolation as you can see here.

howardwu commented 4 years ago

15

howardwu commented 4 years ago

As the original issue was resolved, closing this thread. Feel free to open a new one if there are further issues, thanks for digging into the codebase.