search

m-kovalsky / fabric_cat_tools

Supercharge your Microsoft Fabric development with the fabric_cat_tools library
MIT License
110 stars 15 forks source link

[Question] Post Migration getting error while opening the edit tables #25

Open cvlmonica opened 5 months ago

cvlmonica commented 5 months ago

I have migrated a import mode dataset to direct lake mode ( which has data in Fabric lakehouse). Within it few calculated tables couldn't be migrated. But our ETL has taken care of it in lakehouse building the same table name with data so while I am trying to click on edit tables so that I can select that missing migrated calculated tables but I am getting this error:

image

I have tried multiple times to access but still same error. Even though the semantic model refreshes successful (which means connectivity to that lakehouse is working fine). Any solution around this!?

m-kovalsky commented 5 months ago

Did you try opening the semantic model in Tabular Editor to see if there are any errors in the model? Also make sure that the entity partition name for each table matches the corresponding table in your lakehouse.

cvlmonica commented 5 months ago

yes, I did checked it in the tabular editor , as I have a measure named period_selection which is pointing to a calculated table. And while doing the migration that table is not migrated.

I think if the calculated tables/or any other object which is not migrated then anything related to it should not be migrated to prevent such errors! What is the solution for this!?

image (5)
m-kovalsky commented 5 months ago

I can append the migration process with a step to remove any objects with errors. It's quite simple and you can even run this code to do it for your own model. This code is for measures but the same principle can be applied to any object.

with connect_semantic_model(dataset='DatasetName', readonly=False, workspace='WorkspaceName') as tom:
    for m in tom.all_measures():
        if m.ErrorMessage != '':
            tom.remove_object(object=m)
cvlmonica commented 5 months ago

thank you so much!

cvlmonica commented 5 months ago

I hope you will be adding a step in migration where if the objects has errors those gets removed from the model. And what I observed is, consider I have an import mode dataset and I have couple of calculated tables/columns and I have moved these logic into ETL logic and moved those calculated tables as tables into my lakehouse. When I ran the migration logic and checks for the validation function it shows that a calculated column is not migrated but actually it has been migrated to the direct lake semantic model. Is this a bug or what is it!? can you help is understand this!? Attaching the screenshots below to get a clear understanding: image

image