search

databricks / databricks-sdk-go

Databricks SDK for Go
https://docs.databricks.com/dev-tools/sdk-go.html
Apache License 2.0
51 stars 42 forks source link

[ISSUE] cannot create grants: unexpected error handling request: invalid character '<' looking for beginning of value #752

Open zaxier opened 10 months ago

zaxier commented 10 months ago

Description Bug faced when using databricks_grants resource using databricks terraform provider. Error: cannot create grants: unexpected error handling request: invalid character '<' looking for beginning of value. This is likely a bug in the Databricks SDK for Go or the underlying REST API. Please report this issue with the following debugging information to the SDK issue tracker at https://github.com/databricks/databricks-sdk-go/issues. Request log:

Reproduction

terraform apply
resource "databricks_metastore" "this" {
  name          = "metastore"
  owner         = var.account_admin_group_name
  region        = "ap-southeast-2"
  force_destroy = true
}

resource "databricks_grants" "this" {
  metastore = databricks_metastore.this.id
  grant {
    principal = var.account_admin_group_name
    privileges = ["CREATE_CATALOG", "CREATE_EXTERNAL_LOCATION"]
  }

  # grant {
  #   principal = var.workspace_user_group_name
  #   privileges = ["CREATE_SHARE", "SET_SHARE_PERMISSION", "USE_MARKETPLACE_ASSETS", "USE_CONNECTION", "USE_PROVIDER", "USE_RECIPIENT", "USE_SHARE"]
  # }

  # grant {
  #   principal = var.workspace_admin_group_name
  #   privileges = ["CREATE_EXTERNAL_LOCATION", "CREATE_CONNECTION", "CREATE_PROVIDER", "CREATE_RECIPIENT", "CREATE_SHARE", "CREATE_STORAGE_CREDENTIAL", "MANAGE_ALLOWLIST", "SET_SHARE_PERMISSION", "USE_MARKETPLACE_ASSETS", "USE_CONNECTION", "USE_PROVIDER", "USE_RECIPIENT", "USE_SHARE"]
  # }
}

Expected behavior I expected it to deploy grants.

Is it a regression? Unknown

Debug Logs

│ GET /login?next_url=/api/2.1/unity-catalog/permissions/metastore/927c05a7-42b6-460f-a4bc-913510b835bc
│ > * Host: 
│ > * Accept: application/json
│ > * Authorization: REDACTED
│ > * Referer: https://accounts.cloud.databricks.com/api/2.1/unity-catalog/permissions/metastore/927c05a7-42b6-460f-a4bc-913510b835bc
│ > * User-Agent: databricks-tf-provider/1.32.0 databricks-sdk-go/0.28.1 go/1.20.12 os/linux terraform/1.6.6 resource/grants auth/oauth-m2m
│ < HTTP/2.0 200 OK
│ < * Cache-Control: no-cache, no-store, must-revalidate
│ < * Content-Type: text/html; charset=utf-8
│ < * Date: Thu, 28 Dec 2023 01:24:26 GMT
│ < * Server: databricks
│ < * Strict-Transport-Security: max-age=31536000; includeSubDomains; preload
│ < * X-Content-Type-Options: nosniff
│ < * X-Ui-Svc: true
│ < <!doctype html>
│ < <html lang="en">
│ <  <head>
│ <   <meta charset="utf-8">
│ <   <meta name="viewport" content="width=device-width,initial-scale=1">
│ <   <meta name="theme-color" content="#000000">
│ <   <meta name="description" content="Databricks Sign in">
│ <   <title>Databricks - Sign in</title>
│ <   <script id="Cookiebot" src="https://consent.cookiebot.com/uc.js" data-cbid="459f54ba-f28b-4a56-ab47-7af5ef8b04b4" data-blockingmode="auto" type="text/javascript" defer></script>
│ <   <script>window.__DATABRICKS_CONFIG__={"publicPath":{"accounts-console":"https://ui-assets.cloud.databricks.com/"}}</script>
│ <   <link rel="icon" href="https://ui-assets.cloud.databricks.com/favicon.ico">
│ <   <script defer src="https://ui-assets.cloud.databricks.com/static/js/5005.329e3591.js"></script>
│ <   <script defer src="https://ui-assets.cloud.databricks.com/static/js/4582.62ad0ae9.js"></script>
│ <   <script defer src="https://ui-assets.cloud.databricks.com/static/js/1721.f5335b21.js"></script>
│ <   <script defer src="https://ui-assets.cloud.databricks.com/static/js/9592.cd9e3474.js"></script>
│ <   <script defer src="https://ui-assets.cloud.databricks.com/static/js/7384.ee78b6b9.js"></script>
│ <   <script defer src="https://ui-assets.cloud.databricks.com/static/js/6871.cbb14b60.js"></script>
│ <   <script>
│ <   function setNoCdnAndReload() {
│ <       const secIn7Days = 60 * 60 * 24 * 7;
│ <       document.cookie = `x-databricks-cdn-inaccessible=true; path=/; max-age=${secIn7Days}`;
│ <       const metric = 'cdnFallbackOccurred';
│ <       const browserUserAgent = navigator.userAgent;
│ <       const browserTabId = window.browserTabId;
│ <       const performanceEntry = performance.getEntriesByType('resource').filter(e => e.initiatorType === 'script').slice(-1)[0]
│ <       sessionStorage.setItem('databricks-cdn-fallback-telemetry-key', JSON.stringify({ tags: { browserUserAgent, browserTabId }, performanceEntry}));
│ <       window.location.reload();
│ <   }
│ < </script>
│ <   <script defer src="https://ui-assets.cloud.databricks.com/static/js/login.a950c75d.js" onerror="setNoCdnAndReload()"></script>
│ <   <link href="https://ui-assets.cloud.databricks.com/static/css/4582.c2a40ece.css" rel="stylesheet">
│ <   <link href="https://ui-assets.cloud.databricks.com/static/css/1451.199d10d0.css" rel="stylesheet">
│ <   <link href="https://ui-assets.cloud.databricks.com/static/css/login.3ba2d2e0.css" rel="stylesheet">
│ <  </head>
│ <  <body>
│ <   <noscript>
│ <    You need to enable JavaScript to run this app.
│ <   </noscript>
│ <   <div id="login"></div>
│ <  </body>
│ < </html>

│ │ with module.uc_metastore.databricks_grants.this, │ on modules/4_uc_metastore/_uc_metastore.tf line 9, in resource "databricks_grants" "this": │ 9: resource "databricks_grants" "this" { │

Other Information

Additional context Add any other context about the problem here.

mgyucht commented 10 months ago

Thanks for reporting this. I've followed up with the underlying team to respond with an appropriate error message rather than redirecting to the login page.

However, there is still an underlying issue that will need to be resolved. Are you able to create other account-level resources with your provider?

nkvuong commented 10 months ago

@zaxier databricks_grants is only available as workspace-level resources

jesinity commented 10 months ago

I got pretty much the same error and @nkvuong databricks_grants according to documentation are used as well to grant privileges on metastore resources.

my code snippet is:

resource "databricks_grants" "dbrk_metastore_grant_we" {
  provider  = databricks.account

  metastore = databricks_metastore.we.id
  grant {
    principal  = "metastore admins"
    privileges = [
      "CREATE_EXTERNAL_LOCATION"
    ]
  }
}
nkvuong commented 10 months ago

@jesinity it is counter-intuitive, but you still needs to execute that via a workspace. Essentially, it will grant access to the metastore that is currently assigned to the workspace.

mohitdungrani commented 10 months ago

Hello @nkvuong, I'm currently experiencing a similar issue while attempting to provide access to the metastore via an account. Although I've understood a solution to grant access via a workspace, I'm curious to understand the reasons behind Terraform's inability to accomplish the same access grant directly through an account

sohailshaikh1920 commented 10 months ago

Hello @mohitdungrani , can you help me understand how you authenticated from workspace while deploying unity cataloge with tf?

mohitdungrani commented 9 months ago

Hello @sohailshaikh1920

To perform authentication via workspace, we need to pass the workspace URL inside the host parameter.

For example:

provider "databricks" { host = azure_client_id = var.CLIENT_ID azure_tenant_id = var.TENANT_ID azure_client_secret = var.CLIENT_SECRET }

Reference: https://registry.terraform.io/providers/databricks/databricks/latest/docs#argument-reference