search

eBay / griffin

Model driven data quality service
https://ebay.github.io/griffin/
Other
240 stars 165 forks source link

Asset type only contains hivetable, but how to connect with my hive table? #31

Open ljzhu1990 opened 8 years ago

ljzhu1990 commented 8 years ago 
 Hello author, I have two questions. I want to get your help and study this execllent framework.
 First, In user guide, it describe asset type including hdfsfile and hivetable. But, it only has hivetable when I deploy the ROOT.war to tomcat.
Second. In my opinion, if i select hivetable asset type. It can provide a way to connect with our hive table. But, on web page, it only let us enter a hdfs path. How can I create a userful data asset ? 
Thank you very much!
bhlx3lyx7 commented 8 years ago

Hello ljzhu1990, glad to hear your questions. It seems that you run griffin in your own environment, I'll try to answer your questions here. Griffin is now in version 0.1.0-SNAPSHOT, it was designed to support hdfsfile as well as hivetable at first, we've only implemented hivetable to cover our environment situation now, hdfsfile will be considered in the following version. About your second question, griffin works on spark based on hadoop, we assumed that user deploy griffin engine and hive in the same hadoop cluster, so griffin can find data by hdfs path only. Griffin doesn't support hive remote mode at current, and we'll try to implement it as well. Once again, thank you for your support and questions to help griffin evolve.