Is it possible to create lakehouse shortcuts with schema in both source and destination?

[johanrex]

Is it possible to create lakehouse shortcuts with schema in both source and destination?

This is what I've tried:

    print(f"{table_name=}")
    print(f"{source_lakehouse_name=}")
    print(f"{source_workspace_name=}")
    print(f"{dest_lakehouse_name=}")
    print(f"{dest_workspace_name=}")

    labs.lakehouse.create_shortcut_onelake(
        table_name = table_name, 
        source_lakehouse = source_lakehouse_name, 
        source_workspace = source_workspace_name, 
        destination_lakehouse = dest_lakehouse_name, 
        destination_workspace = dest_workspace_name
        )

Output:

table_name='kmtoll.billing_details_header'
source_lakehouse_name='Silver_Lakehouse'
source_workspace_name='Dev-Silver'
dest_lakehouse_name='Gold_Lakehouse'
dest_workspace_name='Dev-Gold'
---------------------------------------------------------------------------
FabricHTTPException                       Traceback (most recent call last)
File ~/cluster-env/clonedenv/lib/python3.11/site-packages/sempy_labs/lakehouse/_shortcuts.py:73, in create_shortcut_onelake(table_name, source_lakehouse, source_workspace, destination_lakehouse, destination_workspace, shortcut_name)
     72 try:
---> 73     response = client.post(
     74         f"/v1/workspaces/{destinationWorkspaceId}/items/{destinationLakehouseId}/shortcuts",
     75         json=request_body,
     76     )
     77     if response.status_code == 201:

File ~/cluster-env/clonedenv/lib/python3.11/site-packages/sempy/fabric/_client/_rest_client.py:167, in BaseRestClient.post(self, path_or_url, *args, **kwargs)
    149 """
    150 POST request to the Fabric and PowerBI REST API.
    151 
   (...)
    165     The response from the REST API.
    166 """
--> 167 return self.request("POST", path_or_url, *args, **kwargs)

File ~/cluster-env/clonedenv/lib/python3.11/site-packages/sempy/fabric/_client/_rest_client.py:307, in FabricRestClient.request(self, method, path_or_url, lro_wait, lro_max_attempts, lro_operation_name, *args, **kwargs)
    282 """
    283 Request to the Fabric REST API.
    284 
   (...)
    305     The response from the REST API.
    306 """
--> 307 response = super().request(method, path_or_url, *args, **kwargs)
    309 if not lro_wait or response.status_code != 202:

File ~/cluster-env/clonedenv/lib/python3.11/site-packages/sempy/fabric/_client/_rest_client.py:125, in BaseRestClient.request(self, method, path_or_url, *args, **kwargs)
    123 kwargs["headers"] = headers
--> 125 return self.http.request(method, *args, **kwargs)

File ~/cluster-env/clonedenv/lib/python3.11/site-packages/requests/sessions.py:589, in Session.request(self, method, url, params, data, headers, cookies, files, auth, timeout, allow_redirects, proxies, hooks, stream, verify, cert, json)
    588 send_kwargs.update(settings)
--> 589 resp = self.send(prep, **send_kwargs)
    591 return resp

File ~/cluster-env/clonedenv/lib/python3.11/site-packages/requests/sessions.py:710, in Session.send(self, request, **kwargs)
    709 # Response manipulation hooks
--> 710 r = dispatch_hook("response", hooks, r, **kwargs)
    712 # Persist cookies

File ~/cluster-env/clonedenv/lib/python3.11/site-packages/requests/hooks.py:30, in dispatch_hook(key, hooks, hook_data, **kwargs)
     29 for hook in hooks:
---> 30     _hook_data = hook(hook_data, **kwargs)
     31     if _hook_data is not None:

File ~/cluster-env/clonedenv/lib/python3.11/site-packages/sempy/_utils/_log.py:348, in mds_log.<locals>.get_wrapper.<locals>.log_decorator_wrapper(*args, **kwargs)
    347 try:
--> 348     result = func(*args, **kwargs)
    350     # The invocation for get_message_dict moves after the function
    351     # so it can access the state after the method call

File ~/cluster-env/clonedenv/lib/python3.11/site-packages/sempy/fabric/_client/_rest_client.py:65, in BaseRestClient.__init__.<locals>.validate_rest_response(response, *args, **kwargs)
     64 if response.status_code >= 400:
---> 65     raise FabricHTTPException(response)

FabricHTTPException: 400 Bad Request for url: https://api.fabric.microsoft.com//v1/workspaces/e5fb3d3a-5f95-452a-b4ce-3581fb48964e/items/518499c4-4c0f-4e7c-80f4-eaea38ff8647/shortcuts
Error: {"requestId":"8f8baa1f-7e2d-4939-a034-e11b32921fc6","errorCode":"BadRequest","moreDetails":[{"errorCode":"ShortcutNameValidationError","message":"Shortcut name cannot contain the following character(s) : \" \\ / : | < > * ? . % +"}],"message":"The request could not be processed due to missing or invalid information"}
Headers: {'Cache-Control': 'no-store, must-revalidate, no-cache', 'Pragma': 'no-cache', 'Transfer-Encoding': 'chunked', 'Content-Type': 'application/json; charset=utf-8', 'x-ms-public-api-error-code': 'BadRequest', 'Strict-Transport-Security': 'max-age=31536000; includeSubDomains', 'X-Frame-Options': 'deny', 'X-Content-Type-Options': 'nosniff', 'RequestId': '8f8baa1f-7e2d-4939-a034-e11b32921fc6', 'Access-Control-Expose-Headers': 'RequestId', 'request-redirected': 'true', 'home-cluster-uri': 'https://wabi-north-europe-l-primary-redirect.analysis.windows.net/', 'Date': 'Fri, 29 Nov 2024 12:34:39 GMT'}

The above exception was the direct cause of the following exception:

ValueError                                Traceback (most recent call last)
Cell In[43], line 15
     12 print(f"{dest_lakehouse_name=}")
     13 print(f"{dest_workspace_name=}")
---> 15 labs.lakehouse.create_shortcut_onelake(
     16     table_name = table_name, 
     17     source_lakehouse = source_lakehouse_name, 
     18     source_workspace = source_workspace_name, 
     19     destination_lakehouse = dest_lakehouse_name, 
     20     destination_workspace = dest_workspace_name
     21     )

File ~/cluster-env/clonedenv/lib/python3.11/site-packages/sempy_labs/lakehouse/_shortcuts.py:85, in create_shortcut_onelake(table_name, source_lakehouse, source_workspace, destination_lakehouse, destination_workspace, shortcut_name)
     83         print(response.status_code)
     84 except Exception as e:
---> 85     raise ValueError(
     86         f"{icons.red_dot} Failed to create a shortcut for the '{table_name}' table."
     87     ) from e

ValueError: 🔴 Failed to create a shortcut for the 'kmtoll.billing_details_header' table.

[m-kovalsky]

I believe that lakehouse-related APIs do not yet work with schema-enabled lakehouses. I believe this is documented and hopefully will be resolved in the near future.

[johanrex]

It's possible to create a shortcut with schema in both source and destination in the Fabric UI. I.e. exactly the scenario I'm trying to solve with code above. I would have guessed the UI uses the API but perhaps that's not the case.

I did look for any limitations in the API but couldn't find anything mentioned in the documentation: https://learn.microsoft.com/en-us/rest/api/fabric/core/onelake-shortcuts/create-shortcut?tabs=HTTP