Snowflake - Add `data-heap` and `data-test` events column in snowflake.

kedro-org / kedro-viz

Visualise your Kedro data and machine-learning pipelines and track your experiments.

https://demo.kedro.org

Apache License 2.0

680 stars 113 forks source link

Snowflake - Add `data-heap` and `data-test` events column in snowflake. #2175

Open rashidakanchwala opened 2 weeks ago

rashidakanchwala commented 2 weeks ago

Description

Currently, the data-test and data-heap-event attributes, which provide information about the elements users click on, are part of a long string. We currently extract this data using Tableau; however, it would be ideal to have tables created directly in Snowflake. This ticket aims to create a view in Snowflake with additional columns to extract and store this information.

Checklist

[ ] Include labels so that we can categorise your feature request

rashidakanchwala commented 2 weeks ago

Getting this error - SQL execution error: Creating view on shared database 'HEAP_FRAMEWORK_VIZ_PRODUCTION' is not allowed. FYI - @astrojuanlu

astrojuanlu commented 2 weeks ago

The error message says it all, the Heap exports are read-only. We have a KEDRO_BI_DB database with write access (role KEDRO_BI_ROLE, your account has it).

rashidakanchwala commented 2 weeks ago

I tried creating view there, but because it's two different warehouses; it's not allowing me

astrojuanlu commented 2 weeks ago

Then the only way will be having an ETL pipeline... hopefully with Kedro :) similar to

It's a good opportunity to aggregate the information and filter only what we need https://github.com/kedro-org/kedro-devrel/issues/145