search

microsoft / semantic-link-labs

Early access to new features for Microsoft Fabric's Semantic Link.
MIT License
179 stars 41 forks source link

Validate that Source and Destination Dataset values are not the same for Direct Lake Migration in the same workspace #108

Closed thedaxshepherd closed 2 months ago

thedaxshepherd commented 2 months ago

Describe the bug This is due to human error and not sure if this would be an enhancement or a bug. You will get unexpected results if you set the source and destination datasets to the same value (in the same workspace). The error is KeyError: 'Refresh Policy' which can be hard to track down. If you figure out your mistake and correct the destination to a different name, it is too late since the source dataset has already been corrupted. The source dataset needs to be replaced with a copy. Once this happened, I ran through it several times to make sure it was always the same error and whether I could recover.

To Reproduce

  1. I used the Migration to Direct Lake notebook.
  2. Import the library and set initial parameters: Set dataset_name and new_dataset_name to the same value
  3. Import the library and set initial parameters: Make sure both workspace names are the same or set to none which will use current workspace
  4. Create the Direct Lake model based on the import/DQ semantic model: Running this step is what will generate the error

Expected behavior Should report that source and destination datasets cannot be the same and exit.

thedaxshepherd commented 2 months ago

This works great. I understand that I need to add some sanity checks to my copy of the Migration to Direct Lake notebook so that it also validates the inputs. That being said, should create_blank_semantic_model have a check to see if it is overwriting an existing model? If so, should I submit this as an enhancement?

m-kovalsky commented 2 months ago

Yes that is a good idea. I will be adding the overwrite parameter to several functions including this one.

From: Jason Romans @.> Sent: Monday, September 9, 2024 7:29:43 PM To: microsoft/semantic-link-labs @.> Cc: Subscribed @.***> Subject: Re: [microsoft/semantic-link-labs] Validate that Source and Destination Dataset values are not the same for Direct Lake Migration in the same workspace (Issue #108)

This works great. I understand that I need to add some sanity checks to my copy of the Migration to Direct Lake notebook so that it also validates the inputs. That being said, should create_blank_semantic_model have a check to see if it is overwriting an existing model? If so, should I submit this as an enhancement?

— Reply to this email directly, view it on GitHubhttps://github.com/microsoft/semantic-link-labs/issues/108#issuecomment-2338564270, or unsubscribehttps://github.com/notifications/unsubscribe-auth/AHBQBNXUYBY57ESS6MXDBWDZVXEHPAVCNFSM6AAAAABNMX4OJGVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDGMZYGU3DIMRXGA. You are receiving this because you are subscribed to this thread.Message ID: @.***>

m-kovalsky commented 2 months ago

This is fixed in 0.7.3.