search

microsoft / semantic-link-labs

Early access to new features for Microsoft Fabric's Semantic Link.
MIT License
178 stars 37 forks source link

Vertipaq Export #103

Open prathyusha-kamasani opened 2 months ago

prathyusha-kamasani commented 2 months ago

Hi,

Is it possible to update scripts to Export results without attaching the Lakehouse, please? So we can dynamically mount LH and export data into LH tables. Currently, when I mount with script I get below error, it will be good be able to mount dynamically. Many Thanks! Prathy

ValueError: 🔴 In order to save the Vertipaq Analyzer results, a lakehouse must be attached to the notebook. Please attach a lakehouse to this notebook.

m-kovalsky commented 2 months ago

I would like to enable saving to any lakehouse in any workspace however currently it is problematic. Saving the results is not the problem. In order to add the proper 'RunId', I must query the delta table to get the max run Id. Running spark queries cross-workspace is not supported. Thus, using the lakehouse attached to the notebook ensures no problems. Once spark queries can be executed cross-workspace, I'd be happy to update the functionality to support saving to any lakehouse in any workspace.

eisber commented 1 month ago

@m-kovalsky you could use delta-rs to read directly in onelake...

https://blog.fabric.microsoft.com/en-us/blog/delta-rs-introduces-native-support-for-onelake/