sassoftware / viya4-deployment

This project contains Ansible code that creates a baseline in an existing Kubernetes environment for use with the SAS Viya Platform, generates the manifest for an order, and then can also deploy that order into the Kubernetes environment specified.
Apache License 2.0
70 stars 64 forks source link

feat: (IAC-1096) add support for Azure service bus #446

Closed riragh closed 1 year ago

riragh commented 1 year ago

Changes:

Support added for Message broker: Azure service bus to be used by SAS Viya Platform deployment instead of RabbitMQ. The changes to create Azure service bus were added in viya4-iac-azure: PR #320

Variables introduced in this change:

Note: This is currently Experimental/Undocumented feature this will announced along with SAS Viya feature availability.

Name Description Type Default Required Notes Tasks
V4_CFG_MESSAGE_BROKER_ENABLE Enables the message broker created in your cloud provider. Default is set to false which uses RabbitMQ string false false Currently only supported for Azure's message broker: Azure Service Bus viya
V4_CFG_MESSAGE_BROKER_HOST Hostname to access the message broker string false viya
V4_CFG_MESSAGE_BROKER_NAME Specifies the message broker's name string false viya
V4_CFG_MESSAGE_BROKER_PASSWORD The message broker's primary key or password string false viya

Tests:

Verified following on the scenarios listed below, please see internal ticket for details.

Note: SAS Viya Platform services are still not ready. Almost all pods stabilized except for few services. Launcher service is failing with [BROKER_MISSING_USERNAME] causing other services to fail. This is expected until all the services are updated to use the new Azure Service bus instead of Rabbitmq.

Scenario Task Deploy method Cadence
1 Defaults, V4_CFG_MESSAGE_BROKER_ENABLE= true, internal Postgres ansible fast:2020
2 V4_CFG_MESSAGE_BROKER_ENABLE= true, external Postgres ansible fast:2020
3 V4_CFG_MESSAGE_BROKER_ENABLE= true. Values used from tfstate: message_broker_name= "ritika1", aks_cluster_sku_tier = "Standard", external Postgres ansible fast:2020
riragh commented 1 year ago

Same comment as with the other changes. This should have been more generic and cloud agnostic or at least investigated and reasoning why the naming was kept azure specific especially since we'll probably need to handle this on AWS/GCP in the future.

Same reply as with the other repo. I had investigated the different available messaging services across cloud, following are the available options:

Azure: Azure Service Bus: Azure Service Bus is a fully managed enterprise message broker with message queues and publish-subscribe topics (in a namespace).

AWS: Amazon MQ: Amazon MQ is a managed message broker service for Apache ActiveMQ and RabbitMQ that streamlines setup, operation, and management of message brokers on AWS.

GCP: Google Pub/Sub: Pub/Sub is an asynchronous and scalable messaging service that decouples services producing messages from services processing those messages.

All of the above services are fully managed enterprise level message brokers. Hence to keep variables and module names generic I went with name as message broker instead of message queue.

Each message broker has an option to create message queues or message topics. SAS Viya Platform services choose to create Topics. So I thought it would be confusing to name the variables/modules as message queue when we don't create queues.