RabbitMQ: Investigate Queue/Exchange configurations

[dfitchett]

Summary

Current queue and exchange configurations have been captured in this drawio diagram. Each of the exchange and queues need to be investigated and configuration verified, and changes need to be suggested to better fit our use cases.

For example, perhaps it would be better for our BIE kafka messages to be dead lettered to a separate queue, or maybe some queues should not be auto-deleted.

Acceptance Criteria

This ticket should be commented with a table indicating the current configuration, the proposed updates (if any), and the reasoning behind the update. This ticket is for information only, the implementation will be done in the follow up ticket #3330.

Properties that should be captured include:

• Exchanges:
  • type
  • durable
  • autodelete
  • policy
• Queues
  • durable
  • autodelete
  • exclusive (queues only)
  • policy
• Dead Letter Exchanges (DLX)
  • type
  • durable,
  • autodelete
  • policy
• Dead Letter Queues (DLQ)
  • durable
  • autodelete
  • exclusive
  • policy
• OTHER

For situational awareness, make a separate table for any policies that are on the queues. There will have been one added by this https://app.zenhub.com/workspaces/vro-team-6557e67173391c000e1409f3/issues/gh/department-of-veterans-affairs/abd-vro/3327 for max queue limits. Investigate if any other policies would be useful.

[dfitchett]

Created a document showing the current configurations and proposed changes here. The document outlines the above AC, but also lists out some useful definitions and Rules/Guidelines for future implementations of RabbitMQ. This document will be used/updated to modify configurations for svc-bip-api, svc-bgs-api, and ep-merge configurations of queues/exchanges. Additional work will be done to create any policies applied on rabbitmq startup and policies will be deployed by next biweekly deployment of rabbitmq pod.

In addition, follow-on ticket was created from the discussion during post-sprint planning adhoc meeting on 8/27 and is linked below. In summary:

• svc-bie-kafka and xample-workflow add unnecessary complexity for data visibility by splitting the streaming of kafka events and saving the events to the RDS database
• exchange & queues should be reevaluated to have the optimal configuration for this new setup
  • so this documents suggestions for how to fix svc-bie-kafka and xample-workflow configurations will not be addressed in the follow on ticket #3330