search

vesoft-inc / nebula

A distributed, fast open-source graph database featuring horizontal scalability and high availability
https://nebula-graph.io
Apache License 2.0
10.61k stars 1.19k forks source link

Export WAL logs to 3rd-party service, such as Hive #164

Open sherman-the-tank opened 5 years ago

sherman-the-tank commented 5 years ago

We need to export incremental data to 3rd-party service, such as HBase and Hive. One possible solution is to export the WAL logs to a standard format (CSV?) and use utilities such as Flume to dump the logs to the external services

spacewalkman commented 5 years ago

scenarios:

  1. identity incremental data, impose no burden on Nebula's writing performance in the mean time