Add support for UC volumes in DABs

[shreyas-goenka]

Changes

This PR adds the UC volumes resource to DABs. This PR also adds a warning if a user tries to use a volume in the artifact_path that is managed by DABs and has not been deployed yet. Users would need to do two incremental deploys in this case because we always upload artifacts before creating all the resources.

A proper fix where we can allow using the volume within the same deploy would require us to head in a direction of removing our dependency on terraform to manage the CRUD lifecycle of DABs resources.

This PR also has custom behaviour for UC volumes, where presents.name_prefix or the default prefix is only applied on a volume when the catalog or schema is not already scoped to the user.

Tests

Unit, integration test and manually.

Manual outputs:

1. UC volume does not exist:

   ➜  bundle-playground git:(master) ✗ cli bundle deploy
   Error: failed to fetch metadata for the UC volume /Volumes/main/caps/my_volume that is configured in the artifact_path: Not Found

2. UC Volume does not exist, but is defined in DAB

   
   ➜  bundle-playground git:(master) ✗ cli bundle deploy
   Error: failed to fetch metadata for the UC volume /Volumes/main/caps/managed_by_dab that is configured in the artifact_path: Not Found

Warning: You might be using a UC volume in your artifact_path that is managed by this bundle but which has not been deployed yet. Please deploy the UC volume in a separate bundle deploy before using it in the artifact_path. at resources.volumes.bar in databricks.yml:24:7

[shreyas-goenka]

Nightlies passed once. Triggered another round after some minor updates.

[github-actions[bot]]

If integration tests don't run automatically, an authorized user can run them manually by following the instructions below:

Trigger: go/deco-tests-run/cli

Inputs:

• PR number: 1762
• Commit SHA: 1218178e64844a091fb06293b6fc14552273942b

Checks will be approved automatically on success.

[eng-dev-ecosystem-bot]

Test Details: go/deco-tests/11616451005

[witi83]

Is there a potential release date for this feature? We're really looking forward to it

[shreyas-goenka]

@witi83 Thanks for reaching out, this PR is still atleast a week or two from being merged and released.

The primary blocker is us trying to consolidate how we do prefixing for UC resources (with dev_<your-name> for example) when a bundle has mode: development set and we do not want to add support for UC volumes before we have clarity there.

We are actively working on unblocking and merging this PR.