search

DataThirstLtd / azure.databricks.cicd.tools

Tools for Deploying Databricks Solutions in Azure
MIT License
97 stars 63 forks source link

Deploy Databricks files to DBFS task donot work with PLE enabled workspace #209

Open Sjain01 opened 9 months ago

Sjain01 commented 9 months ago

I was able to use this task until the time my workspace was not behind private link endpoint. But quite recently it started throwing an error when I enabled private endpoint. The error is as follows-

Tools Version: 2.2.5299 2023-12-15T14:17:55.2138906Z <!doctype html> 2023-12-15T14:17:55.2139824Z <html> 2023-12-15T14:17:55.2142146Z <head> 2023-12-15T14:17:55.2142570Z <meta charset="utf-8"> 2023-12-15T14:17:55.2143211Z <meta http-equiv="Content-Language" content="en"> 2023-12-15T14:17:55.2143848Z <title>Databricks - Sign In</title> 2023-12-15T14:17:55.2144496Z <meta name="viewport" content="width=960"> 2023-12-15T14:17:55.2145634Z <link rel="icon" type="image/png" href="https://databricks-ui-assets.azureedge.net/favicon.ico"> 2023-12-15T14:17:55.2146645Z <meta http-equiv="content-type" content="text/html; charset=UTF8"> 2023-12-15T14:17:55.2147635Z <script id="__databricks_react_script"></script> 2023-12-15T14:17:55.2151095Z <script>window.__DATABRICKS_SAFE_FLAGS__={"databricks.infra.showErrorModalOnFetchError":true,"databricks.fe.infra.useReact18":false},window.__DATABRICKS_CONFIG__={"publicPath":{"mlflow":"https://databricks-ui-assets.azureedge.net/","dbsql":"https://databricks-ui-assets.azureedge.net/","feature-store":"https://databricks-ui-assets.azureedge.net/","monolith":"https://databricks-ui-assets.azureedge.net/","jaws":"https://databricks-ui-assets.azureedge.net/"}}</script> 2023-12-15T14:17:55.2154622Z <link rel="icon" href="https://databricks-ui-assets.azureedge.net/favicon.ico"> 2023-12-15T14:17:55.2155452Z <script> 2023-12-15T14:17:55.2155848Z function setNoCdnAndReload() { 2023-12-15T14:17:55.2156418Z const msIn1Day = 1000 * 60 * 60 * 24 * 1; 2023-12-15T14:17:55.2157220Z document.cookie =x-databricks-cdn-inaccessible=true; path=/; max-age=\${msIn1Day}; 2023-12-15T14:17:55.2158053Z const metric = 'cdnFallbackOccurred'; 2023-12-15T14:17:55.2158667Z const browserUserAgent = navigator.userAgent; 2023-12-15T14:17:55.2159323Z const browserTabId = window.browserTabId; 2023-12-15T14:17:55.2160309Z const performanceEntry = performance.getEntriesByType('resource').filter(e => e.initiatorType === 'script').slice(-1)[0] 2023-12-15T14:17:55.2162502Z sessionStorage.setItem('databricks-cdn-fallback-telemetry-key', JSON.stringify({ metric, tags: { browserUserAgent, browserTabId }, performanceEntry})); 2023-12-15T14:17:55.2165236Z window.location.reload(); 2023-12-15T14:17:55.2165757Z } 2023-12-15T14:17:55.2166129Z </script> 2023-12-15T14:17:55.2167096Z <script defer src="https://databricks-ui-assets.azureedge.net/static/js/login/login.4611ec12.js" onerror="setNoCdnAndReload()"></script> 2023-12-15T14:17:55.2168249Z </head> 2023-12-15T14:17:55.2168691Z <body class="light-mode"> 2023-12-15T14:17:55.2169247Z <uses-legacy-bootstrap> 2023-12-15T14:17:55.2169801Z <div id="login-page"></div> 2023-12-15T14:17:55.2170340Z </uses-legacy-bootstrap> 2023-12-15T14:17:55.2170875Z </body> 2023-12-15T14:17:55.2171296Z </html>

My suspicion is this is because it calls generic databricks rest api uri - https://westeurope.azuredatabricks.net/api/2.0/dbfs/put to execute this activity. Is there a way to make use of specific url of databricks in place of this generic one ?

I tested it via postman , when provided generic uri as stated above , it doesnt work. However when I specified workspace url and executed the script it worked without any issues.

i3bdallah commented 7 months ago

@Sjain01 @simondmorias is there any solution to this issue ? we are facing the same issue with the Databricks Task in the Marketplace, same error