vesoft-inc / nebula-importer

Nebula Graph Importer with Go
Apache License 2.0
90 stars 60 forks source link

Support value template in props definition #287

Open xiajingchun opened 1 year ago

xiajingchun commented 1 year ago

Is your feature request related to a problem? Please describe.

Right now, only "index" is supported in props definition to map a prop value to a column of csv. Sometimes, this requires a lot of prework to format the csv files.

Describe the solution you'd like

Probably, we can add a "template" to allow more flexible value mapping. Say, $i refers to the i th column in the csv.

The simplest template is like below, same as "index";

  props:
    - name: "propname"
      type: "xxxxx"
      template: $i

But how about below?

tempalte: left($i, 3) tempalte: $i + $j tempalte: timestamp(datetime($i)) tempalte: case when $i==something then $j else $k end

You can think of more.

Basically, most of the nGQL functions and also "case when" can be supported in the templates. This will use more GraphD CPU, but the usability gain is a lot.

Describe alternatives you've considered

Additional context

veezhang commented 1 year ago

@MuYiYong It feels like the requirements need to be refined:

  1. If there is a template, will other fields take effect? For example: nullable, nullValue, alternativeIndices, defaultValue. If they take effect, what is the order?
  2. How to deal with types in templates? For example, should $i be a string or a number?
  3. What to do if $ is not followed by a number?
  4. What to do if the template itself contains the $ character?
  5. After starting the template, do I still need to perform type conversion? For example, whether left($i, 3) still needs quotation marks, and another example, such as prefix_$0 seems need add quotation marks.
image
xiajingchun commented 1 year ago

my 2 cents:

  1. template should definitely override index, and other null related settings should be ingored too.
  2. no need to deal with the data type, let the user decide, and in this case, it will be "$i" when $i is a string, e.g., left("$i",3)