Guidelines to compile and execute

[mencagli]

Very nice project. I would like to try to use your code. Can you please help in understanding how to compile the individual projects (for example, Flink/Storm/Spark benchmarks) and run them?

Many thanks

Gabriele

[ShuhaoZhangTony]

Thanks for your interest! The whole project is managed by maven. In order to include Flink, Storm benchmarks, you only need to enable the corresponding maven module in the root pom file. Spark benchmark is outdated, and I haven't got time to update it.

[mencagli]

Thanks for your answer. However, I have found some problems:

1-When I start "mvn package" on the main folder, I received an error "Child module /home/guest/briskstream/Java-Thread-Affinity-master of /home/mencagli/briskstream/pom.xml does not exist". The problem is that the module should be named "Java-Thread-Affinity" and not "Java-Thread-Affinity-master". This is easy to fix by editing your main pom.xml;

2-Compilation of the briskstream benchmarks is ok, although I don't understand how to run the experiments and to configure their settings. Some indications will be highly appreciated :)

3-There are problems to compile the FlinkBenchmarks module (I guess also for Storm). Here a list of problems in the source code: --error "'parent.relativePath' of POM org.tony:Flink:1.3.2 (/home/guest/briskstream/FlinkBenchmarks/pom.xml) points at org.tony:briskstream instead of org.tony:brisk". I guess the problem is that in the pom.xml (flink folder) we have "brisk" as artifactId -> it should be "briskstream". Easy to fix by editing pom.xml; --error in file "FlinkBenchmarks/src/main/java/applications/bolts/vs/ACDBolt.java". At line 52 a comment is missing in that line. Easy to fix by changing by hand your source code file; --I am not able to fix this final error. I have received many errors after running meaven because it seems that some java files in the application "lr" inside the FlinkBenchmarks folder are not able to access the package "applications.bolts.lr.data" which is totally missing.

I hope you can help me

Thanks for the patience,

Gabriele

[mencagli]

Furthermore, these are the first errors that I receive. As it is evident, it seems that there are some missing packages to compile FlinkBenchmarks

[ERROR] /home/guest/briskstream/FlinkBenchmarks/src/main/java/applications/bolts/Executor.java:[19,43] package applications.bolt.helper does not exist [ERROR] /home/guest/briskstream/FlinkBenchmarks/src/main/java/applications/bolts/lr/DailyExpenditureBolt.java:[23,34] package applications.bolts.lr.data does not exist [ERROR] /home/guest/briskstream/FlinkBenchmarks/src/main/java/applications/bolts/lr/DailyExpenditureBolt.java:[24,34] package applications.bolts.lr.data does not exist [ERROR] /home/guest/briskstream/FlinkBenchmarks/src/main/java/applications/bolts/lr/DailyExpenditureBolt.java:[25,39] package applications.bolts.lr.data.toll does not exist [ERROR] /home/guest/briskstream/FlinkBenchmarks/src/main/java/applications/bolts/lr/DailyExpenditureBolt.java:[26,39] package applications.bolts.lr.data.toll does not exist [ERROR] /home/guest/briskstream/FlinkBenchmarks/src/main/java/applications/bolts/lr/DailyExpenditureBolt.java:[27,39] package applications.bolts.lr.data.util does not exist [ERROR] /home/guest/briskstream/FlinkBenchmarks/src/main/java/applications/bolts/lr/DailyExpenditureBolt.java:[28,39] package applications.bolts.lr.data.util does not exist [ERROR] /home/guest/briskstream/FlinkBenchmarks/src/main/java/applications/bolts/lr/DailyExpenditureBolt.java:[65,27] cannot find symbol

[ShuhaoZhangTony]

Thanks for letting me know!

I have just checked with the project and confirmed the problem -- the dependency is somehow broken when I export the repo from bitbucket to github. It is fixed now, and you can try to run each sub-project, BriskStream, Storm, Flink by simply launch the corresponding Runner, e.g., BriskBenchmarks/src/main/java/applications/BriskRunner.java. This will run the project with the default local configuration without NUMA-awareness. There are scripts in common module to run Storm and Flink in cluster model, and scripts for running BriskStream in different configurations as well.

For your information, the following class contains information to specify different input configurations so you can prepare your own scripts. briskstream/common/src/main/java/applications/abstractRunner.java

Stay tuned that I'll try to update a more informative readme later at my best efforts, but currently busy with some other on-going projects.

[mencagli]

Thanks for your prompt reply. Unfortunately, some problems are still there. I clone the project from scratch and try "mvn package". Still not problem for briskstreamBenchamarks but impossible to compile Flink Benchmarks. There are missing packages:

[ERROR] /home/mencagli/briskstream/FlinkBenchmarks/src/main/java/applications/bolts/lr/DailyExpenditureBolt.java:[23,34] package applications.bolts.lr.data does not exist [ERROR] /home/mencagli/briskstream/FlinkBenchmarks/src/main/java/applications/bolts/lr/DailyExpenditureBolt.java:[24,34] package applications.bolts.lr.data does not exist [ERROR] /home/mencagli/briskstream/FlinkBenchmarks/src/main/java/applications/bolts/lr/DailyExpenditureBolt.java:[25,39] package applications.bolts.lr.data.toll does not exist [ERROR] /home/mencagli/briskstream/FlinkBenchmarks/src/main/java/applications/bolts/lr/DailyExpenditureBolt.java:[26,39] package applications.bolts.lr.data.toll does not exist [ERROR] /home/mencagli/briskstream/FlinkBenchmarks/src/main/java/applications/bolts/lr/DailyExpenditureBolt.java:[27,39] package applications.bolts.lr.data.util does not exist [ERROR] /home/mencagli/briskstream/FlinkBenchmarks/src/main/java/applications/bolts/lr/DailyExpenditureBolt.java:[28,39] package applications.bolts.lr.data.util does not exist [ERROR] /home/mencagli/briskstream/FlinkBenchmarks/src/main/java/applications/bolts/lr/DailyExpenditureBolt.java:[65,27] cannot find symbol symbol: class TollDataStore

[mencagli]

As you can see, in the folder "FlinkBenchmarks/src/main/java/applications/bolts/lr/" there is no "data" subfolder and so impossible to compile. Probably something has been forgotten during the commit. Thanks :)

[ShuhaoZhangTony]

Same issue there. Solved. :) The package is actually there but in different path. I'm not sure what causes it. I'm guessing it's due to the auto code-clean features of Intellij IDEA, which I used once during repo migration.

[mencagli]

Thanks for your support. Unfortunately, it is impossible to compile FlinkBenchmarks and StormBenchmarks. There are several broken packages because the structure of the folders downloaded by github does not respect what is expected to compile properly. You can try to clone the project from scratch in your machine and to compile it to discover the problems. Probably for you, that will for sure have a local fully working copy of the project, it would be relatively easy to fix it. Thanks again

[ShuhaoZhangTony]

Fixed and tested. It's again the same issue, path is updated wrongly during code clean, so those affected classes cannot find the header file. Sorry for my careless mistake. I have successfully compiled it in a clean machine so it should work now. Thanks for pointing out the problem!

[mencagli]

Thank you so much. I successfully compiled FlinkBenchmarks. I will play with them. Many many thanks for your hard work:)