search

RedisLabs / spark-redis-ml

A spark package for loading Spark ML models to Redis-ML
http://redisml.io
Other
63 stars 14 forks source link

Does not build with original Spark ML library #4

Open shry15harsh opened 7 years ago

shry15harsh commented 7 years ago

I get the error of missing getImpurityStats() method in Spark ML package. I checked the Dockerfile because in Docker it was running fine, then I found out that modified Spark distribution is used there in which this method has been added. The pull request and the corresponding issue is still in progress on Spark JIRA. Nowhere in the steps, it is mentioned. Also, RedisLabs has written a lot about Spark-Redis integration on its website and Databricks gave presentation and tutorial. So I am wondering why nobody discussed it or is it just me for whom it is breaking?

shaynativ commented 7 years ago

Good point. On earlier versions the impurity stats weren't required. I'll try to speed up the pull request on Spark and meanwhile add this point the documentation.

shry15harsh commented 7 years ago

Thanks. It would be really awesome if it gets in Spark. It would be really a scalable and fast model serving solution.