search

microsoft / semantic-link-labs

Early access to new features for Microsoft Fabric's Semantic Link.
MIT License
178 stars 37 forks source link

Extract full table source information from semantic model #245

Closed jaskier2 closed 2 weeks ago

jaskier2 commented 3 weeks ago

Is your feature request related to a problem? Please describe. Not related to a problem.

Describe the solution you'd like I would like to extract the information on where the semantic model is getting its data from. i.e. a Fabric SQL Analytics Endpoint or a different SQL Server etc.

I was able to get the list of semantic model objects and I see object types of table but it isn't enough info to trace back where that table is located.

I also was able to get the model.bim and after poking around a bit I found the info located in model > [ ] tables > { } table_num > [ ] partitions { } partition_num.

However, it doesn't seem consistent between model.bim's as another model had the info located in model > [ ] tables > { } table_num > [ ] partitions { } partition_num > { } source > [ ] expression.

Describe alternatives you've considered Parsing the .bim, but this seems difficult, wondering if there is a better way.

Additional context Ultimately, we are looking to intelligently refresh the semantic model when new data has arrived in one of the source tables in that semantic model.

m-kovalsky commented 3 weeks ago

You can easily extract the partition info using semantic link:

'''python import sempy.fabric as fabric

fabric.list_partitions(dataset='', workspace='') '''

Also you do not need to parse the semantic model metadata as there is the TOM API for that.

https://semantic-link-labs.readthedocs.io/en/stable/sempy_labs.tom.html

jaskier2 commented 2 weeks ago

Thank you!