huawei-noah / streamDM

Stream Data Mining Library for Spark Streaming
http://streamdm.noahlab.com.hk/
Apache License 2.0
492 stars 147 forks source link

Update File Reader. Add limit instance as stopping condition #72

Closed nhnminh closed 7 years ago

nhnminh commented 7 years ago

This pull request has 1 change in File Reader:

Change:

Reason: Stopping Condition, used for benchmarking (computing running time).

Test:

We could run the code as follow:

./spark.sh "200 EvaluatePrequential  -l (trees.HoeffdingTree -l 0 -t 0.05 -g 200 -o) -s (FileReader -f ../data/electNormNew.arff -k 4000 -d 10 -i 30000)"  1> result.res 2> log.log

Here, we limit the number of instances to be 30000