Add a mechanism to load Python source locations in the Python mutator. Previously, locations pointed to generated YAML. Now, they point to Python sources instead. Python process outputs "locations.json" containing locations of bundle paths, examples:
Such locations form a tree, and we assign locations of the closest ancestor to each dyn.Value based on its path. For example, resources.jobs.job_0.tasks[0].task_key is located at job_0.py:10:5 and resources.jobs.job_0.tasks[0].email_notifications is located at job_0.py:3:5, because we use the location of the job as the most precise approximation.
This feature is not yet enabled by default because PyDABs don't yet ignore unknown arguments, and this will be only supported with 0.6.0.
Note: for now, we don't update locations with relative paths, because it has a side effect in changing how these paths are resolved
Example
% databricks bundle validate
Warning: job_cluster_key abc is not defined
at resources.jobs.examples.tasks[0].job_cluster_key
in src/examples/example.py:10:1
Changes
Add a mechanism to load Python source locations in the Python mutator. Previously, locations pointed to generated YAML. Now, they point to Python sources instead. Python process outputs "locations.json" containing locations of bundle paths, examples:
Such locations form a tree, and we assign locations of the closest ancestor to each
dyn.Value
based on its path. For example,resources.jobs.job_0.tasks[0].task_key
is located atjob_0.py:10:5
andresources.jobs.job_0.tasks[0].email_notifications
is located atjob_0.py:3:5
, because we use the location of the job as the most precise approximation.This feature is not yet enabled by default because PyDABs don't yet ignore unknown arguments, and this will be only supported with 0.6.0.
Note: for now, we don't update locations with relative paths, because it has a side effect in changing how these paths are resolved
Example
Tests
Unit tests and manually
Stacked on top of https://github.com/databricks/cli/pull/1782