search

databricks / cli

Databricks CLI
Other
132 stars 50 forks source link

Add validation for files with a `.(resource-name).yml` extension #1780

Open shreyas-goenka opened 1 week ago

shreyas-goenka commented 1 week ago

Changes

We want to encourage a pattern of only specifying a single resource in a YAML file when an .(resource-type).yml (like .job.yml) is used. This convention could allow us to bijectively map a resource YAML file to it's corresponding resource in the Databricks workspace.

This PR:

  1. Emits a recommendation diagnostic when we detect this convention is being violated. We can promote this to a warning when we want to encourage this pattern more strongly.
  2. Visualises the recommendation diagnostics in the bundle validate command.

Tests

Unit tests and manually. Here's what an example output looks like:

Recommendation: define a single job in a file with the .job.yml extension.
  at resources.jobs.bar
     resources.jobs.foo
  in foo.job.yml:13:7
     foo.job.yml:5:7

The following resources are defined or configured in this file:
  - bar (job)
  - foo (job)
shreyas-goenka commented 5 days ago

Triggered nightlies against this PR...

shreyas-goenka commented 5 days ago

Nightlies are green!

shreyas-goenka commented 3 days ago

@fjakobs The templates have already been addressed in https://github.com/databricks/cli/pull/1777. I'll also be letting mlops-stacks know about this.