cloudera-labs / cdpy

A Simple Pythonic Client wrapper for Cloudera CDP CLI, designed for use with the Ansible framework
Apache License 2.0
10 stars 22 forks source link

Improve Azure deployment stability #26

Closed Chaffelson closed 3 years ago

Chaffelson commented 3 years ago

No collateral changes to collections

cdpy now recognises that CML deployments may fail during provisioning and responds accordingly cdpy will now automatically retry Azure cross-account credential creation in CDP when receiving eventual consistency errors

Meta-changelist: Fix formatting in cloud.datahub_template_info Add retry to address intermittent failure in datahub_template_info listing of datahub templates in CDP 7.2.10 Add explicit test for Azure Storage Account being unavailable for use in this deployment remove duplicate usage of __azure_netapp_vol_info as variable Handle edge cases for preparation of Netapp NFS Mount when deploying ML automatically on Azure Introduce wait and retry controls for handling Azure eventual consistency when negotiating between Ansible Controller, Azure Control Plane, and CDP Control Plane Move default Azure minimal policy json from private gist to cloudera-labs snippets Correct Azure App name where sometimes referred to with http:// header and sometimes not, resulting in idempotence failures. Oops. Introduce more robust validations that Service principals and other objects created by az CLI are populated as expected Ensure that Azure objects are consistently bound to the Azure namespace created from the name_prefix Provide more user friendly errors when Azure App and Service Principal creation doesn't go as planned Add tunnel and public endpoint control support to Azure Environment creation in line with AWS offering Fix ML submission preparation to include nfs information following existing combination patterns Swap order of Runtime initialization tasks to handle provider-specific tasks before general tasks, to allow Azure-specific values to be populated. No impact on other implementations Explicitly derive Azure NFS Mount information in Runtime deployment from earlier Infrastructure deployment steps Fix purge teardown edgecase where child services are not deleted if at least one child service is not present in the Definition. Now they are removed if purge is set and any are found regardless of provided Definition plan.

Signed-off-by: Daniel Chaffelson chaffelson@gmail.com