"add_field_parameter" removes option to edit model in service

[benjimanupfd]

Describe the bug When using "add_field_paramerer" function the model in the service shows error symbols against the new table. The model then does not have the ability to be edited by the service and throws an error.

To Reproduce Steps to reproduce the behavior:

1. Run tom.add_field_parameter against semantic model using Fabric notebook https://github.com/m-kovalsky/fabric_cat_tools?tab=readme-ov-file#adds-a-field-parameter-to-a-semantic-model
2. Got to model in Fabric service.
3. "Open data model" to view the data model in the service.
4. Error shown against newly created field parameters table.
5. Click "Edit tables" in the ribbon.
6. "Fetching schema" is shown and after ~30s an error message is shown.

Expected behavior No error message should be shown agains tthe field parameter table. No error should be occur when clicking "Edit tables" and list of connected lakehouse tables should be shown.

Screenshots

Desktop (please complete the following information):

• OS: Win11
• Browser: Firefox & Edge (same issue in both)
• Power BI Service Version 13.0.23515.57
• CAT Tools v0.4.1

Additional context MS ticket raised. Response from support engineers are external tools (CAT tools included) do not support field parameters or any schema modification to semantic models.

[m-kovalsky]

I identified the bug and fixed it (within 0.4.1). Please delete the field parameter you created and recreate it using this function using the 0.4.1 I just released.

[benjimanupfd]

I removed and reinstalled the latest version (0.4.1), upadted my env in Fabric and restarted the session but still have the same issues - including error message in the py script: "Max retries exceeded with url ... Caused by ResponseError('too many 500 error responses'"

I also tested witha brand new semantic model (just to be sure) and same issues: Error on field params:

Edit semantic model error:

If it helps, adding field parameters seemed to take a very long time relative to other functions (e.g. 10mins instead of seconds for 2 fields). This is likely related to the retry error though.

[m-kovalsky]

Did you get any messages after running the add_field_parameter function? It should indicate that the semantic model has been refreshed. Also make sure you are using the proper syntax for measures [Measure Name] and columns ‘Table Name’[Column Name].

---

From: benjimanupfd @.> Sent: Friday, June 14, 2024 4:04:51 PM To: m-kovalsky/fabric_cat_tools @.> Cc: Michael Kovalsky @.>; Comment @.> Subject: Re: [m-kovalsky/fabric_cat_tools] "add_field_parameter" removes option to edit model in service (Issue #26)

I removed and reinstalled the latest version (0.4.1), upadted my env in Fabric and restarted the session but still have the same issues - including error message in the py script: "Max retries exceeded with url ... Caused by ResponseError('too many 500 error responses'"

I also tested witha brand new semantic model (just to be sure) and same issues: Error on field params: image.png (view on web)https://github.com/m-kovalsky/fabric_cat_tools/assets/104636060/81aa0c9a-96dc-4035-b32f-d64c9649b9fe

Edit semantic model error: image.png (view on web)https://github.com/m-kovalsky/fabric_cat_tools/assets/104636060/cd7ba80d-73ef-4278-b479-7b924a78f060

If it helps, adding field parameters seemed to take a very long time relative to other functions (e.g. 10mins instead of seconds for 2 fields). This is likely related to the retry error though.

— Reply to this email directly, view it on GitHubhttps://github.com/m-kovalsky/fabric_cat_tools/issues/26#issuecomment-2167999384, or unsubscribehttps://github.com/notifications/unsubscribe-auth/AHBQBNTOLNSTE6RL2CAOO33ZHLS7HAVCNFSM6AAAAABJKBKUWSVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDCNRXHE4TSMZYGQ. You are receiving this because you commented.Message ID: @.***>

[benjimanupfd]

No messages about model refresh - the error is thrown first (after ~10 mins) However if I edit the DAX for the field params after it's created (whtout actually changing anything) then I can get the error icons to disappear on the field params. I assume this triggers a refresh of the model manually from the service UI.

However, the subsequent issue of not fetching lakehouse schema still applies.

If it helps - here's the script I'm running (nothing sensitive): with connect_semantic_model(dataset = 'test_sm', workspace = 'D2I_Gold', readonly = False) as tom: tom.add_field_parameter( table_name = 'params_rows', objects = ["d2i_dim_material[category]","d2i_dim_customer[desc_level1]"] ) print(f'Field Param added')

[m-kovalsky]

It appears the field parameter is successfully created. After it’s created it triggers a refresh. For some reason with your model the refresh part is taking a long time and not working. Seems like something you should investigate. The field parameter will only work (and not show errors) if a refresh is done on that table. As I mentioned the function takes care of it but there appears to be an issue with your model as the refresh is not working.

---

From: benjimanupfd @.> Sent: Friday, June 14, 2024 4:32:16 PM To: m-kovalsky/fabric_cat_tools @.> Cc: Michael Kovalsky @.>; Comment @.> Subject: Re: [m-kovalsky/fabric_cat_tools] "add_field_parameter" removes option to edit model in service (Issue #26)

No messages about model refresh - the error is thrown first (after ~10 mins) However if I edit the DAX for the field params after it's created (whtout actually changing anything) then I can get the error icons to disappear on the field params. I assume this triggers a refresh of the model manually from the service UI.

However, the subsequent issue of not fetching lakehouse schema still applies.

If it helps - here's the script I'm running (nothing sensitive): image.png (view on web)https://github.com/m-kovalsky/fabric_cat_tools/assets/104636060/917f65c5-9703-4da0-8630-8f21b86124d2 with connect_semantic_model(dataset = 'test_sm', workspace = 'D2I_Gold', readonly = False) as tom: tom.add_field_parameter( table_name = 'params_rows', objects = ["d2i_dim_material[category]","d2i_dim_customer[desc_level1]"] ) print(f'Field Param added')

— Reply to this email directly, view it on GitHubhttps://github.com/m-kovalsky/fabric_cat_tools/issues/26#issuecomment-2168061921, or unsubscribehttps://github.com/notifications/unsubscribe-auth/AHBQBNQS36ZSJSIFMWEJOBTZHLWGBAVCNFSM6AAAAABJKBKUWSVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDCNRYGA3DCOJSGE. You are receiving this because you commented.Message ID: @.***>

[benjimanupfd]

So after many tests via a variety of different setups we discovered some caching issues in a browser was causing semantic model not to refresh after adding field params.

However, the main issue still exists (last step in original bug) - when trying to "Edit tables" in the semantic model (using UI) we get the error message. This has been tested across computers, browser, Fabric capacities, workspaces, and lakehouses. It only happens after adding a field parameter - and if we remove the field paramtere it works immediately again.

Once field param added - can we no longer interact with the model via UI?

[m-kovalsky]

I presume this is because calculated tables are technically not supported so it is breaking this operation. However, as you have shown you can still open the data model in the web and make changes. Are you trying to add a new table to your semantic model? If so, you can do this via TOM. I think I'll also add a function for Direct Lake models which allows for selecting a specific lakehouse table and having that table added to the semantic model (with all of the columns and their properties). In that case, you wouldn't need the option to 'Edit Tables'.

[benjimanupfd]

Yes - I believe this is the case.

If you could add the function to add tables to a semantic model from a lakehouse that would be perfect - and correct - we wouldn't need to edit models in the service anymore and the above error would not be an issue. Please let me know when you've added and I can be a tester if you need one!

[m-kovalsky]

Great. I've written the function but no further changes will be made to fabric_cat_tools. This library is being moved here and will be released soon. Once the new library is released I will publish the function in that library.

[benjimanupfd]

Sounds good - I'll keep my eye out for it. Thanks for all the responses.

[m-kovalsky]

Please try the add_table_to_direct_lake_semantic_model function in semantic link labs. It was released yesterday in 0.6.0