KryoSerializer doesn't work in LDADriver

cloudml / zen

Zen aims to provide the largest scale and the most efficient machine learning platform on top of Spark, including but not limited to logistic regression, latent dirichilet allocation, factorization machines and DNN.

Apache License 2.0

170 stars 75 forks source link

KryoSerializer doesn't work in LDADriver #25

Closed bhoppi closed 8 years ago

bhoppi commented 9 years ago

It is done.

hucheng commented 9 years ago

Great. Please share here the root cause analysis and the corresponding fix.

bhoppi commented 8 years ago

The Spark conf setting "spark.kryo.classesToRegister" does not work. We call:

val conf: SparkConf
conf.registerKryoClasses(Array(classOf[BSV], classOf[BDV], ...))

which works.