search

amplab / velox-modelserver

http://amplab.github.io/velox-modelserver/
Apache License 2.0
110 stars 26 forks source link

Add tachyon back as dependency #28

Open dcrankshaw opened 9 years ago

dcrankshaw commented 9 years ago

PR #27 removed tachyon as a dependency to get binary compatibility between hadoop-client and spark-core. Even though we can't use the Tachyon KV store right now, we can still use Tachyon as a faster version of HDFS to transfer training data and models between Spark and Velox. We should add tachyon back as a storage option once we can figure out class compatibility between Spark, Hadoop, and Tachyon.

tomerk commented 9 years ago

As of pull request #44 this may be as simple as changing the spark data location config to be in the tachyon filesystem