Open kfot opened 1 year ago
The problem lies in the handling of the dbx
default values in the dbfs and repo functions.
The profile
param value defaults to "DEFAULT"
string
def(
...,
profile: str = PROFILE_OPTION,
...,
):
(because the PROFILE_OPTION
defined here references the databricks-cli
default profile value defined here...)
Then, within the dbfs and repo
functions (two identical code sections), the value is passed to the get_databricks_config function (which has misleading profile=None
signature) and then in the body the function considers our "DEFAULT"
profile value as a valid, explicitly provided profile which it is clearly not.
if profile:
config = ProfileConfigProvider(profile).get_config() <- our guy ends up here
else:
config = get_config() <- while it should go right here
Is there any progress on resolving this? This bug makes it very difficult to use dbx sync
in a CICD setting, and cost significant time troubleshooting before discovering this issue.
Expected Behavior
Make the
dbx sync dbfs
respects the same standard variables as thedatabricks-cli
and make it consistent with its own tutorial describing usage ofDATABRICKS_HOST
andDATABRICKS_TOKEN
env variables as an authentication alternative to.databrickscfg
file.Current Behavior
Having sourced valid
DATABRICKS_HOST
andDATABRICKS_TOKEN
into the shell and the.databrickscfg
file missing, the command raises following error.Your Environment