vesoft-inc / nebula-importer

Nebula Graph Importer with Go
Apache License 2.0
90 stars 60 forks source link

Add option to output DML lines #289

Open wey-gu opened 1 year ago

wey-gu commented 1 year ago

Recently I wanted to transfer my CSV files into ngql DML lines, so I quickly changed as follows to do so.

I wonder if someone else has such a requirement, too, If yes, maybe I could PR to add an option to output to log instead of stdout.

@veezhang do you think this feature may be reasonable or not?

diff --git a/pkg/spec/v3/edge.go b/pkg/spec/v3/edge.go
index 71c771a..e88664c 100644
--- a/pkg/spec/v3/edge.go
+++ b/pkg/spec/v3/edge.go
@@ -193,7 +193,11 @@ func (e *Edge) Validate() error {
 }

 func (e *Edge) Statement(records ...Record) (statement string, nRecord int, err error) {
-       return e.fnStatement(records...)
+       statement, nRecord, err = e.fnStatement(records...)
+       if err == nil && statement != "" {
+               fmt.Println("Generated command:", statement)
+       }
+       return statement, nRecord, err
 }

 func (e *Edge) insertStatement(records ...Record) (statement string, nRecord int, err error) {
diff --git a/pkg/spec/v3/node.go b/pkg/spec/v3/node.go
index 2c3929e..b084028 100644
--- a/pkg/spec/v3/node.go
+++ b/pkg/spec/v3/node.go
@@ -147,7 +147,11 @@ func (n *Node) Validate() error {
 }

 func (n *Node) Statement(records ...Record) (statement string, nRecord int, err error) {
-       return n.fnStatement(records...)
+       statement, nRecord, err = n.fnStatement(records...)
+       if err == nil && statement != "" {
+               fmt.Println("Generated command:", statement)
+       }
+       return statement, nRecord, err
 }

 func (n *Node) insertStatement(records ...Record) (statement string, nRecord int, err error) {
veezhang commented 1 year ago

@wey-gu You just want to store the statement to a file instead of storing in nebula.?

wey-gu commented 1 year ago

@wey-gu You just want to store the statement to a file instead of storing in nebula.?

Yes, I generated a small dataset as CSV and was preparing for studio starter dataset, so I did this with above change :-P

Not sure how general this requirement could be to other users.

veezhang commented 1 year ago

@wey-gu So, why not use nebula-importer tool to import directly?

  1. generated csv to ngql
  2. use studio to exec ngql

=>

  1. use nebula-importer import csv
wey-gu commented 1 year ago

@wey-gu So, why not use nebula-importer tool to import directly?

  1. generated csv to ngql
  2. use studio to exec ngql

=>

  1. use nebula-importer import csv

Yes, for pure import purposes, that's enough, only just this use case I had to prepare files for this

Maybe this isn't a general requirement at all.

veezhang commented 1 year ago

@wey-gu Maybe studio should use csv as datasets instand of ngql? csv has the following advantages: