search

tokern / data-lineage

Generate and Visualize Data Lineage from query history
https://tokern.io/data-lineage/
MIT License
308 stars 44 forks source link

Support for Google BigQuery #11

Open ranugoldan opened 4 years ago

ranugoldan commented 4 years ago

Any possibility to look out about Google BigQuery? BigQuery already have query history feature, and we can retrieve it from bigquery logs exported table.

vrajat commented 4 years ago

The main hurdle is that there is no parser for BigQuery. I came across this reddit thread and ZetaSQL. ZetaSQL seems like a good building block. I would love some help in adding support. Are you open to collaborating on building this feature?

terekete commented 2 years ago

has there been any further work on integrating BigQuery ?

vrajat commented 2 years ago

You can process BigQuery queries using this example There will be a lot parsing failures as this app is using a Postgres parser.

As mentioned in my previous comment, google/zetasql provides a parser for Google Bigquery. I am looking for support (contributions/collaboration or financial) to support it.

pulkitsri1729 commented 2 years ago

Hi @vrajat I am open for contribution for this feature

rjt-moonpay commented 1 year ago

Any update on this feature?