yanboliang / spark-vlbfgs

Vector-free L-BFGS implementation for Spark MLlib
Apache License 2.0
46 stars 17 forks source link

Add VectorSummarizer & improve VLOR aggregating code #11

Closed WeichenXu123 closed 7 years ago

WeichenXu123 commented 7 years ago
  1. add VectorSummarizer

  2. improve VLOR aggregating vectors code, replace reduceByKey with aggregateByKey using VectorSummarizer

  3. In VBinomialLogisticCostFun before shuffle vectors, call Vector.compressed to reduce shuffling data size.

  4. add licenses for codes.