The example currently doesn't have a private link setup included as indicated in the architecture diagram. The PR adds an optional private link along with other changes based on field experience.
Changes:
Added missing private link setup. Added variable to enabled/disabled the private link. By default disabled since it requires the Databricks Enterprise tier so won't work for all customers.
Added ready configuration for each region (webapp, SCC relay, hive metastore, control plan). The configuration can still be provided explicitly as before. This greatly simplifies the setup for the customers.
Updated Readme
Added output.tf so that users don't have to do it
Upgraded provider (databricks, aws) versions
Whitelisted repositories for maven and spark packages
Updated aws services setup as per the latest aws provider version
Refactored in places and provided better comments
Deployment tested with and without private link enabled
The example currently doesn't have a private link setup included as indicated in the architecture diagram. The PR adds an optional private link along with other changes based on field experience.
Changes: