search

open-data-products / odp

Toolkit for understanding the value of your data
Other
8 stars 3 forks source link

With --grain=table, catalog (DB) is sometimes missing #25

Open dexhorthy opened 6 months ago

dexhorthy commented 6 months ago

Some will have a catalog:

DEX_DEV.INFORMATION_SCHEMA.COLUMNS: 43
DEX_DEV.DEX_DEV.ISSUES: 39

But for others its missing

.DEX_DEV.ISSUES: 37
.LINEAR.ISSUES: 29

Maybe something is up w/ qualifying the database name from the query connection?

$ python odp/cli/main.py detect-unused  --grain table --since-days 180
.INFORMATION_SCHEMA.COLUMNS: 819
DEX_DEV.LINEAR.ISSUES: 164
SNOWFLAKE.ACCOUNT_USAGE.QUERY_HISTORY: 79
PUBLIC.LINEAR.ISSUES: 50
DEX_DEV.INFORMATION_SCHEMA.COLUMNS: 43
DEX_DEV.DEX_DEV.ISSUES: 39
.DEX_DEV.ISSUES: 37
.LINEAR.ISSUES: 29
DEX_DEV.LINEAR.CYCLES: 23
.DEX_DEV.WORKFLOWSTATES: 19
DEX_DEV.DEX_DEV.WORKFLOWSTATES: 18
DEX_DEV.LINEAR.WORKFLOWSTATES: 17
.DEX_DEV.CYCLES: 17
PUBLIC.LINEAR.WORKFLOWSTATES: 16
.DEX_DEV.USERS: 14
.LINEAR.WORKFLOWSTATES: 13
PUBLIC.INFORMATION_SCHEMA.COLUMNS: 12
DEX_DEV.DEX_DEV.CYCLES: 11
DEX_DEV.LINEAR.USERS: 8
PUBLIC.LINEAR.CUMULATIVE_FLOW: 8
Unused tables (1):
('DEX_DEV', 'DEX_DEV', 'FAKE')
dangoldin commented 5 months ago

Hmm. Can you share the queries?