apache / incubator-wayang

Apache Wayang(incubating) is the first cross-platform data processing system.
https://wayang.incubator.apache.org/
Apache License 2.0
184 stars 73 forks source link

Multicontext features #429

Closed damik3 closed 5 months ago

zkaoudi commented 5 months ago

The online checks give this error: Error: Failed to execute goal net.alchim31.maven:scala-maven-plugin:4.8.1:testCompile (compile-scala) on project wayang-api-scala-java: Execution compile-scala of goal net.alchim31.maven:scala-maven-plugin:4.8.1:testCompile failed: Compilation failed: InterfaceCompileFailed -> [Help 1] Error:
Error: To see the full stack trace of the errors, re-run Maven with the -e switch. Error: Re-run Maven using the -X switch to enable full debug logging. Error:
Error: For more information about the errors and possible solutions, please read the following articles: Error: [Help 1] http://cwiki.apache.org/confluence/display/MAVEN/PluginExecutionException Error:
Error: After correcting the problems, you can resume the build with the command Error: mvn -rf :wayang-api-scala-java Error: Process completed with exit code 1.

damik3 commented 5 months ago

Did a tweak, I think its working now

zkaoudi commented 5 months ago

Got this error now:

Error: Failed to execute goal org.apache.maven.plugins:maven-surefire-plugin:3.0.0-M5:test (default-test) on project wayang-tests-integration: There are test failures. Error:
Error: Please refer to /home/runner/work/incubator-wayang/incubator-wayang/wayang-tests-integration/target/surefire-reports for the individual test results. Error: Please refer to dump files (if any exist) [date].dump, [date]-jvmRun[N].dump and [date].dumpstream. Error: -> [Help 1] Error:
Error: To see the full stack trace of the errors, re-run Maven with the -e switch. Error: Re-run Maven using the -X switch to enable full debug logging. Error:
Error: For more information about the errors and possible solutions, please read the following articles: Error: [Help 1] http://cwiki.apache.org/confluence/display/MAVEN/MojoFailureException Error:
Error: After correcting the problems, you can resume the build with the command Error: mvn -rf :wayang-tests-integration Error: Process completed with exit code 1.

Any idea?

damik3 commented 5 months ago

Yes, the error changed, now it's

org.apache.wayang.core.api.exception.WayangException: Executing T[JavaCollect[convert out@JavaMap[1->1, id=17a8194f]]] failed.
    at org.apache.wayang.tests.TensorflowIrisIT.test(TensorflowIrisIT.java:153)
Caused by: java.lang.IllegalArgumentException: Comparison method violates its general contract!
    at org.apache.wayang.tests.TensorflowIrisIT.test(TensorflowIrisIT.java:153)

I'm not sure why the TensorflowIrisIT test would break. I didn't make any relevant changes. Does the test have any degree of unpredictability in it? I ran it a few times locally, and at first it failed but then without changing anything, it would pass.

I did a new commit with some irrelevant cleanup, maybe we should try the checks again?

zkaoudi commented 5 months ago

Great! It worked now. But before merging one more thing. Can we rename the BlossomContext to WayangContext? Does it contain more things than the WayangContext?

damik3 commented 5 months ago

@zkaoudi Yes BlossomContext extends WayangContext, so they can't have the same name, we have to choose something different. Maybe something like MultiWayangContext or MultiCapableWayangContext because the thing is that BlossomContext still is only one context, but it is the class that should be used for multi context operations instead of the original wayangcontext.

zkaoudi commented 5 months ago

How about WayangMultiContext? Anyone else preferences or ideas? @kbeedkar ?

kbeedkar commented 5 months ago

Yes, WayangMulticontext, or simply multicontext is also fine.

zkaoudi commented 5 months ago

Yes, let's keep it simple: MultiContext. @damik3 could you please make the corresponding changes?

damik3 commented 5 months ago

Just made the changes :)

zkaoudi commented 5 months ago

Thanks a lot Michalis :)

damik3 commented 5 months ago

My pleasure :)