Open KeltonKarboviak opened 3 months ago
@KeltonKarboviak Thank you for opening this (and the draft PR)! One of the concerns I have with your draft PR is the performance impact for accessing another metadata table as well as the fact that AWS has been recommending that we migrate dependencies off of the pg tables for a bit now. Would you be able to check for performance impact in the PR?
I'm curious of what you think of the alternative approach of tagging query history would also meet your needs?
Is this your first time submitting a feature request?
Describe the feature
I want to be able to capture the query ID of the statement used to load a model.
This will be helpful if we ever need to troubleshoot or diagnose a particular model load by taking the query ID and cross-referencing it in the various system tables that give stats & metrics for every query that is run.
Describe alternatives you've considered
I am unsure of an alternative for how to capture the exact query ID that was run for a model besides capturing it in the adapter and providing it in the Adapter Response, which will then save it to the JSON artifacts.
Who will this benefit?
This will be helpful for database administrators, data engineers, and data analysts in needing to troubleshoot or diagnose performance issues with a particular dbt build.
Are you interested in contributing this feature?
Yes
Anything else?
Reference to the
dbt-snowflake
adapter where they are capturingquery_id
in the adapter response: https://github.com/dbt-labs/dbt-snowflake/blob/f95b9192f6eec9af4e30eaab87f9e3412febf7d1/dbt/adapters/snowflake/connections.py#L456-L461