search

cloudml / zen

Zen aims to provide the largest scale and the most efficient machine learning platform on top of Spark, including but not limited to logistic regression, latent dirichilet allocation, factorization machines and DNN.
Apache License 2.0
170 stars 75 forks source link

Prefer VertexRDD over VertexRDDImpl #56

Closed benmccann closed 6 years ago

bhoppi commented 8 years ago

Thank you, but I prefer a full solution that hides EdgeRDDImpl and VertexRDDImpl at the same time, otherwise keep both of them. But EdgeRDDImpl is hard to optimize by only using provided EdgeRDD APIs.

benmccann commented 8 years ago

Yes, EdgeRDDImpl is harder to hide, which is why I left it. But also because you only need to hide VertexRDDImpl in order to use the upstream graphx implementation, so you will get most of the benefit from just hiding VertexRDDImpl whereas hiding EdgeRDDImpl doesn't accomplish much.