microsoft / dbt-fabric

MIT License
79 stars 28 forks source link

[FEATURE] catalog generation should include rowcounts of models where possible #230

Open dataders opened 1 month ago

dataders commented 1 month ago

source: community Slack thread

If the data platform provides rowcounts as a column in the resultset of information_schema.tables, dbt adapters are able to fetch this information via dbt docs generate and store it in the catalog.json, so that it is available in the OSS dbt docs as well as dbt Cloud's Explore product.

one example is dbt-snowflake, which uses the snowflake__get_catalog_tables_sql() macro to fetch Row Count.

from @prdpsvs:

Unfortunately, [Fabric's information_schema] does not [contain rowcount info]. There are multiple ways to retrieve data from sys views/objects but none of them are 100%. Two approaches can work but both are workarounds and not worth the effort but its upto developers.

  • Create stats and query row count using DBCC SHOW statistics
  • Use select count(1)

Perhaps one of these workarounds could be shipped in the adapter, but @prdpsvs would know best if doing so would: