Investigate BigETL project for usefull data scheme and harvesting features

mgalas / Scheme4Data

To perform data audit or generate meta-data on the datasets imported with ETL4Data we want to create a standard data scheme capable of supporting all data types listed in Landscape4Data

GNU General Public License v3.0

1 stars 3 forks source link

Investigate BigETL project for usefull data scheme and harvesting features #4

Open mgalas opened 7 years ago

mgalas commented 7 years ago

BigETL was pointed out as complementary project we can consider when developing data scheme and harvesting features. We want to investigate the project and check if:

a generalized data scheme is available in BigETL that can be applied to Open Street Map dataset we want to use as a toy example
extensions/plugins for Sqoop are available that would speed up our process of developing Sqoop harvester for Open Steet Map

ulince commented 7 years ago

After reviewing the paper and the project's code on Github, I found that while it has a similar idea to our project, there is no data harvesting code or generic data scheme provided.