Are there code repo for dbx, instead of only providing as a wheel

databrickslabs / cicd-templates

Manage your Databricks deployments and CI with code.

Other

202 stars 100 forks source link

Are there code repo for dbx, instead of only providing as a wheel #25

Closed dexianta closed 3 years ago

renardeinside commented 3 years ago

Hi @dexianta , the repository with dbx is currently in private mode, but we have a plan to make it open-sourced as well. I would say that timeline is the end of 2020, but this could be stretched due to the review process.

dexianta commented 3 years ago

Hi @renardeinside , thank you for the reply, and thank you for the big revamp of the previous code!

I'm seeing that in the dbx code, under utils/common.py at line 131, 1.2 version of the rest api is used instead of 2.0, may I ask what's the motivation behind?

Thank you

renardeinside commented 3 years ago

V1.2 API is used only for dbx execute, the reason is that we need to execute some code on an interactive cluster, without creating a new job to avoid package version override across multiple users of the same cluster. For this purpose, we're using execution context API methods.