search

dacort / metabase-athena-driver

An Amazon Athena driver for Metabase 0.32 and later
Apache License 2.0
225 stars 32 forks source link

Athena data source continuously trigger unexpected queries #104

Closed weiyu1zhu closed 1 year ago

weiyu1zhu commented 2 years ago

We successfully connected to Athena and it works. However, Metabase continuously run queries on all the tables, including large tables we have. Here's an example: -- Metabase SELECT "alb_logs"."vprovision"."elb_status_code" AS "elb_status_code" FROM "alb_logs"."vprovision" GROUP BY "alb_logs"."vprovision"."elb_status_code" ORDER BY "alb_logs"."vprovision"."elb_status_code" ASC LIMIT 5000 It scans much data (could be several TB), cost us a lot; and even if we cancel these queries, it seems to rerun them. I've turned off the scan field values and changed sync to daily. Is there anyway to stop such queries on Athena data source?

dacort commented 2 years ago

Hi @weiyu1zhu - this is a feature of Metabase so it can be more efficient in determining field types/cardinality. I'm not sure if it can be entirely disabled, but can look into it - did changing either of the settings help or is it still running the queries?

weiyu1zhu commented 2 years ago

Thanks for the response. It's still running the queries, not sure if they are left over from previous schedule. As I understand, turning off the filter values scan should stop these queries, the sync is only about schema change and should be lightweight.

abramovi commented 1 year ago

Hey @weiyu1zhu did you manage to stopped those query ?

dacort commented 1 year ago

Closing as part of cleanup now that Athena is officially supported by Metabase. Any future issues can be asked about on their forum or with a detailed bug report.