[ISSUE] AccountClient doesn't use host in .databrickscfg

[shannonlowder]

Description When you create a WorkspaceClient, you can set the host in your config file, which will be read successfully. If you define a host for an AccountClient, the host setting is ignored, and a workspace host is selected instead. Since a host gets set, the library code refuses to overwrite an existing value for the host.

Reproduction Create a config file with at least two profiles. One to be used for a WorkspaceClient, and one for an AccountClient.

[account]
host=https://accounts.azuredatabricks.net/
account_id=...
azure_tenant_id=...
azure_client_id=...
azure_client_secret=...

[workspace_one]
host=https://adb-....azuredatabricks.net/
token=...
jobs-api-version=2.0

Then, create an AccountClient using the account profile. You'll find the client uses your workspace host instead.

Expected behavior I would expect the host setting from the account config to be used in my account client.

Debug Logs DEBUG:databricks.sdk:Loaded from environment INFO:databricks.sdk:loading BPCS_Account_Config profile from ~/.databrickscfg: host, account_id, azure_tenant_id, azure_client_id, azure_client_secret DEBUG:databricks.sdk:Ignoring pat auth, because azure-client-secret is preferred DEBUG:databricks.sdk:Ignoring basic auth, because azure-client-secret is preferred DEBUG:databricks.sdk:Ignoring metadata-service auth, because azure-client-secret is preferred DEBUG:databricks.sdk:Ignoring oauth-m2m auth, because azure-client-secret is preferred DEBUG:databricks.sdk:Attempting to configure auth: azure-client-secret INFO:databricks.sdk:Configured AAD token for Service Principal (...) DEBUG:databricks.sdk.oauth:Retrieving token for ... DEBUG:urllib3.connectionpool:Starting new HTTPS connection (1): login.microsoftonline.com:443 DEBUG:urllib3.connectionpool:https://login.microsoftonline.com:443 "POST /.../oauth2/token HTTP/1.1" 200 1437 DEBUG:databricks.sdk.oauth:Retrieving token for ... DEBUG:urllib3.connectionpool:Starting new HTTPS connection (1): login.microsoftonline.com:443 DEBUG:urllib3.connectionpool:https://login.microsoftonline.com:443 "POST /.../oauth2/token HTTP/1.1" 200 1485 DEBUG:urllib3.connectionpool:Starting new HTTPS connection (1): adb-....azuredatabricks.net:443 DEBUG:urllib3.connectionpool:https://**adb-....azuredatabricks.net**:443 "GET /api/2.0/accounts/../scim/v2/Users?filter=userName+co+...%22 HTTP/1.1" 403 321

Other Information

• OS: Windows 11
• Version: 0.6.0,0.7.0

Additional context You can define the host again in your code to get the Account Client to use the correct host, but doesn't that defeat the purpose of the config file?

[pietern]

I suspect something is setting the host before it is loaded from the profile.

Do you happen to have DATABRICKS_HOST configured in your environment?

[pietern]

You can print the config attribute of the client to display where the configuration attributes come from.

[shannonlowder]

DATABRICKS_HOST isn't defined in this environment; I checked the user and machine environment variables and the script.

When I explore the config attribute, I see where it picks up the azure_client_id, tenant_id, client secret, and account id.. Oddly, the host and hostname appear as the last values in my .databrickscfg file.

With further exploration, the host seems to be set by the debugger. A file ./.databricks/.databricks.env is setting the value to the wrong URL. Now, I can fix the issue by removing/altering this file when debugging.

[shannonlowder]

If you have a bad URL entered that doesn't match what you expect, and you're debugging in VSC, check the.databricks folder in your solution. You'll find a env file that is overriding your machine/user environment variables.