StavPonte11 / json-column-parser

JSON parser for several data sources columns
MIT License
4 stars 0 forks source link

Add support to other data sources parsing #4

Open StavPonte11 opened 2 years ago

StavPonte11 commented 2 years ago

Starting with trino (prestosql) and delta

noxify commented 2 years ago

Hi @StavPonte11 ,

i think this "just" a wrapper.

I was able to parse the presto structure with the latest main version with this code snippet:

const parseFields = (
  elements: { name: string; type: string; description: string }[],
) => {
  const parsed = elements.map((element) => {
    const columnParser = {}

    let custom_source = element.type.replace(new RegExp(', ', 'g'), ',')
    custom_source = custom_source.replace(new RegExp(' ', 'g'), ':')

    return parser.parseColumnType(custom_source)
  })

  return parsed
}

Here an example what elements could be: ( we fetch the table schema/columns via https://github.com/tagomoris/presto-client-node#readme and convert the result to the structure below )

const elements = [
  {
    name: "simple",
    type: "varchar",
    description: ""
  },
  {
    name: "complex",
    type: "array(row(sub1 array(varchar), sub2 varchar, sub3 varchar, sub4 varchar, sub5 varchar, sub6 varchar, name varchar, sub7 array(varchar), sub8 boolean, sub9 varchar, sub10 boolean))",
    description: ""
  }
]

Maybe it helps :)

If there are any questions or something else, feel free to give me a ping :)