[bug] fabric_lakehouse resource not working with Service Principal Secret authentication

[ottojaaskelainen]

🐛 What happened?

Currently fabric_lakehouse resource can't be deployed when using Service Principal Client Secret authentication. I'm quite sure the reason is that wrong scopes are used when generating a token for SP to be consumed in the API calls under the hood.

Currently when trying to deploy, first it gives you an error:

"Could not create resource: An unexpected error occurred while processing the request"

And if you try second time:

Could not create resource: Requested '{lakehouse_name}' is already in use

Anyways the lakehouse does not get created and after a while, you can again try with the same name and you end up to the first error. The name is just cached for a while somewhere as reserved.

I have been playing around with Fabric REST APIs and had this same issue when was using wrong scope for token generation. The correct ones to use are:

"https://api.fabric.microsoft.com/Workspace.ReadWrite.All/.default" and "https://api.fabric.microsoft.com/Item.ReadWrite.All/.default"

After using those I'm able to create lakehouses through Service Principal authentication

🔬 How to reproduce?

No response

🏗️ Code Sample / Log

No response

📷 Screenshots

No response

📈 Expected behavior

No response

🌌 Environment (Provider Version)

0.1.0-beta.4

🌌 Environment (Terraform Version)

1.9.7

🌌 Environment (OS)

macOS

📎 Additional context

No response

🔰 Code of Conduct

• [X] I agree to follow this project's Code of Conduct.

[ottojaaskelainen]

Same issue seems to be in place also with azure cli authentication.

[DariuszPorowski]

Hi @ottojaaskelainen Scopes are used only for the user authentication, not for the Service Principals: https://github.com/microsoft/terraform-provider-fabric/blob/main/docs/guides/auth_app_reg_user.md

For the Service Principal you do not setting scopes in the app registration. Below guides will help you correctly configure Entra App for client/secret authentication:

• How to setup app for SPN auth: https://github.com/microsoft/terraform-provider-fabric/blob/main/docs/guides/auth_app_reg_spn.md
• Authenticating using a Service Principal and Client Secret https://github.com/microsoft/terraform-provider-fabric/blob/main/docs/guides/auth_spn_secret.md

[ottojaaskelainen]

The authentication with SP is working. For example workspace, workspace_role and notebook resources are working with SP authentication. But the Lakehouse resource it’s not as I described. What I meant was that because I had similar experience when I was playing with Fabric REST API, the scope that I was passing for the token generation was problematic with Lakehouse create API but worked with all other APIs that had SP authentication supported. So just tried to point out that it might be the issue also with this Lakehouse Terraform resource. Anyways the Lakehouse resource is not working with SP Client Secret authentication.

[DariuszPorowski]

@ottojaaskelainen may you share your hcl terraform code for the lakehouse? I cannot repo this issue.

[ottojaaskelainen]

Hcl for lakehouse creation:

`resource "fabric_lakehouse" "this" { display_name = var.lakehouse_name description = var.lakehouse_description workspace_id = var.workspace_id

configuration = { enable_schemas = var.enable_schemas } } `

And like said, on exactly same template, workspace, workspace role assignment and notebook resource creation goes through. The sp configured for the provider is the owner of the created resources and assigned automatically as Admin for the workspace.

But with the Lakehouse it ends up to the errors described before.

[DariuszPorowski]

Thank you @ottojaaskelainen

I identified the bug. The problem was not in the SPN auth, but when configuration.enable_schemas value was false (with true works)

Failing configuration:

resource "fabric_lakehouse" "example" {
  display_name = "example"
  description  = "example"
  workspace_id = "00000000-0000-0000-0000-000000000000"

  configuration = {
    enable_schemas = false
  }
}

To create lakehouse without schemas currently, the working hcl is skipping configuration attribute.

resource "fabric_lakehouse" "example" {
  display_name = "example"
  description  = "example"
  workspace_id = "00000000-0000-0000-0000-000000000000"
}

The fix is on the way: https://github.com/microsoft/terraform-provider-fabric/pull/47 and will be addressed in the next release.

[ottojaaskelainen]

Hey, that's true, can create lakehouses without configuration attribute! Great that the origin of issue was found !