OpenLineage Adapter

Overview

Guideline to extract lineage info in OpenLineage format from Snowflake ACCESS_HISTORY view.

Code Deployment

OPENLINEAGE_ACCESS_HISTORY View

View Defintion

open_lineage_access_history.sql is the script to create the view from ACCESS_HISTORY and QUERY_HISTORY that outputs each query that accesses tables in the account in OpenLineage JsonSchema specification.

• The view only shows a query that has non-empty value for query_tag column in the query_history.
• The namespace of each record is in the format of snowflake://<Organization_name>-<Account_name>

Prerequisite

Set your account's organization name to the session variable current_organization before creating the view and running each query on the view.

Example

set current_organization='my_org';