renardeinside
commented
3 years ago Hi @azachar !
We're going to OSS dbx in the very soonish future (most probably this quarter). For now, you can install dbx via pip from package and access all source code via introspection in your IDE, for example, PyCharm.
We need some adjustments for a proper CI/CD of the package, PyPI repo, etc.
For instance, how would you get the "deployed full path in databricks aka dbfs://Shared/Projects/xxxxx/yourlibname" out of the "dbx deploy" command?
There is currently no explicit way to get it out of dbx deploy.
What you can do is the following - after performing dbx deploy you can get the deployment path via the following snippet in Python (it's part of of dbx launch:
from dbx.utils.common import (
dbx_echo, generate_filter_string, prepare_environment, environment_option, parse_multiple,
)
environment = "your-environment-name"
prepare_environment(environment) # will authenticate you in MLflow
filter_string = generate_filter_string(environment, additional_tags)
runs = mlflow.search_runs(filter_string=filter_string, max_results=1)
run_info = runs.iloc[0].to_dict()
print(run_info) # will contain the deployment path
Hello,
We would like play with the bundles dbx tool, is there any source available?
For instance, how would you get the "deployed full path in databricks aka dbfs://Shared/Projects/xxxxx/yourlibname" out of the "dbx deploy" command?
Is there a way to pass / export a custom path to dbx deploy command.
Thank you!
Best regards, Andrej