Avoid multiple file tree traversals on bundle deploy

databricks / cli

Databricks CLI

Other

115 stars 39 forks source link

Avoid multiple file tree traversals on bundle deploy #1493

Closed shreyas-goenka closed 2 weeks ago

shreyas-goenka commented 2 weeks ago

Changes

To run bundle deploy from DBR we use an abstraction over the workspace import / export APIs to create a filer.Filer and abstract the file system. Walking the file tree in such a filer is expensive and requires multiple API calls. This PR remove the two duplicate file tree walks that happen by caching the result.