search

microsoft / semantic-link-labs

Early access to new features for Microsoft Fabric's Semantic Link.
MIT License
178 stars 37 forks source link

Unable to update column info in semantic model via update_column function #158

Closed blaine12100 closed 1 month ago

blaine12100 commented 2 months ago

Describe the bug

Trying to update columns in a semantic model via the update_colum function does not work

To Reproduce Steps to reproduce the behavior:

  1. Create a direct lake semantic model like : labs.directlake.generate_direct_lake_semantic_model(dataset=dataset, lakehouse_tables=['mtcars_source', 'mtcars_dupe'], lakehouse='test', overwrite=True, lakehouse_workspace=workspace, workspace=workspace)
  2. Connect the semantic model via the connect_semantic model function in the tom package with labs.tom.connect_semantic_model(dataset=dataset, readonly=False, workspace=workspace) as tom:
  3. Use the update_column function like : tom.update_column(table_name='mtcars_dupe', column_name='model', description='Dummy description to test the working of this function mpg')

Expected behavior The update_column should update the necessary properties for a column in the semantic model without error.

Screenshots image

Additional context Temporary Workaround is to modify the column attributes via a for loop to check if a column exists for a particular table, modify it's description, Ishidden attribute, formatting options etc. Not the recommended approach as it's very tedious. The output is surprising as the column exists for that given table and I'm able to modify some properties via the temp workaround.

m-kovalsky commented 2 months ago

See the ArgumentException at the end of the error message. There is such column 'mtcars_dupe'[model] in your model. Cannot update a column if it doesn't exist.

blaine12100 commented 2 months ago

I've double checked this. The column exists in the lakehouse tables and also in the semantic model. I still get the same error message. If you see the add_relationship function, I'm able to create the relationship by using the model column in both tables (mtcars_source and mtcars_dupe). If the column didn't exist, the add_relationship function would have failed first.

m-kovalsky commented 2 months ago

Ah, I found the bug. I made a PR with the fix.

blaine12100 commented 2 months ago

Will this be merged now or in a later release?

m-kovalsky commented 2 months ago

Will go into next week’s release.

From: Dhruv Marwha @.> Sent: Thursday, September 19, 2024 12:09:02 PM To: microsoft/semantic-link-labs @.> Cc: Michael Kovalsky @.>; Comment @.> Subject: Re: [microsoft/semantic-link-labs] Unable to update column info in semantic model via update_column function (Issue #158)

Will this be merged now or in a later release?

— Reply to this email directly, view it on GitHubhttps://github.com/microsoft/semantic-link-labs/issues/158#issuecomment-2360440542, or unsubscribehttps://github.com/notifications/unsubscribe-auth/AHBQBNS3LA626JZY6QBRT3LZXKIC5AVCNFSM6AAAAABONE3PRKVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDGNRQGQ2DANJUGI. You are receiving this because you commented.Message ID: @.***>

m-kovalsky commented 1 month ago

Fixed in 0.8.0.

blaine12100 commented 1 month ago

I did test it. It does indeed work. Thank you for fixing this. Quick question: I cannot find an option to set the Is Nullable field via the update_column function. Are there plans to enable the user to set it via this function?

m-kovalsky commented 1 month ago

You can also update any such property using TOM directly (see code below). The update functions are for convenience (and mostly for setting properties such as data types which would require TOM.DataType.Int64 as an example.

with connect_semantic_model(dataset='', workspace='', readonly=False) as tom:
    c = tom.model.Tables[''].Columns['']
    c.IsNullable = False
blaine12100 commented 1 month ago

You can also update any such property using TOM directly (see code below). The update functions are for convenience (and mostly for setting properties such as data types which would require TOM.DataType.Int64 as an example.

with connect_semantic_model(dataset='', workspace='', readonly=False) as tom:
    c = tom.model.Tables[''].Columns['']
    c.IsNullable = False

When the functions were not working, this is the approach that I've used. I'll use that only then.

m-kovalsky commented 1 month ago

I’ll add this property to the update_column function for the next release

From: Dhruv Marwha @.> Sent: Tuesday, October 1, 2024 4:12:56 PM To: microsoft/semantic-link-labs @.> Cc: Michael Kovalsky @.>; State change @.> Subject: Re: [microsoft/semantic-link-labs] Unable to update column info in semantic model via update_column function (Issue #158)

You can also update any such property using TOM directly (see code below). The update functions are for convenience (and mostly for setting properties such as data types which would require TOM.DataType.Int64 as an example.

with connect_semantic_model(dataset='', workspace='', readonly=False) as tom: c = tom.model.Tables[''].Columns[''] c.IsNullable = False

When the functions were not working, this is the approach that I've used. I'll use that only then.

— Reply to this email directly, view it on GitHubhttps://github.com/microsoft/semantic-link-labs/issues/158#issuecomment-2386087851, or unsubscribehttps://github.com/notifications/unsubscribe-auth/AHBQBNS7FATM5XUTIIO7MODZZKUWRAVCNFSM6AAAAABONE3PRKVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDGOBWGA4DOOBVGE. You are receiving this because you modified the open/close state.Message ID: @.***>