search

telefonicaid / fiware-cygnus

A connector in charge of persisting context data sources into other third-party databases and storage systems, creating a historical view of the context
https://fiware-cygnus.rtfd.io/
GNU Affero General Public License v3.0
64 stars 105 forks source link

sql2hdfs connector #23

Closed frbattid closed 9 years ago

frbattid commented 10 years ago

As feedback got from Sevilla's FI-WARE event: it would be desirable a component able to dump all the data within a SQL batabase into Cosmos HDFS.

This could start a new kind of "dumping" components which must be disabled once all the data has been moved to Cosmos.

frbattid commented 10 years ago

I did not realize this is something already developed by Apache...

http://sqoop.apache.org/

Sqoop is a tool designed to transfer data between Hadoop and relational databases. You can use Sqoop to import data from a relational database management system (RDBMS) such as MySQL or Oracle into the Hadoop Distributed File System (HDFS), transform the data in Hadoop MapReduce, and then export the data back into an RDBMS.

I'll try to install it in our Global Instance.