osism / issues

This repository is used for bug reports that are cross-project or not bound to a specific repository (or to an unknown repository).
https://www.osism.tech
1 stars 1 forks source link

[bug] not all RabbitMQ feature flags are enabled after OSISM 8.0.0 update #1179

Open maliblatt opened 2 weeks ago

maliblatt commented 2 weeks ago

OSISM release version

8.0.0

What's the problem?

After upgrading to OSISM 8.0.0 everything seems to be running fine. On one of our regions we see sometimes RabbitMQ message timeouts, that is why I took a deeper look into the RabbitMQ. Here I found that not all feature flags have been enabled after the update to 8.0.0. As far as I know after finishing the update all (stable) feature flags have to be enabled. Normally this should be done by

- name: Enable all stable feature flags
  command: "{{ kolla_container_engine }} exec rabbitmq rabbitmqctl enable_feature_flag all"
  become: true
  changed_when: false

But for some reason that has not been applied. In the WebUI I see:

⚠ All stable feature flags must be enabled after completing an upgrade. [[Learn more]](https://www.rabbitmq.com/feature-flags.html)

From CLI:

$ rabbitmqctl list_feature_flags | grep disabled
detailed_queues_endpoint    disabled
khepri_db   disabled
message_containers  disabled
message_containers_deaths_v2    disabled
quorum_queue_non_voters disabled
stream_filtering    disabled
stream_update_config_command    disabled

Is there anybody who can confirm that? Would it be safe and a good idea to issue rabbitmqctl enable_feature_flag all ?

References to existing reports

related upstream commit: https://review.opendev.org/c/openstack/kolla-ansible/+/922607

Severity

low

Urgency

low

berendt commented 2 weeks ago

It looks different in the testbed with the 2024.1 tag. It's fine that khepri_db is disabled, it's an experimental feature. Since the task is always executed, I think it should be ok to execute it manually.

TASK [rabbitmq : Enable all stable feature flags] ******************************
Monday 11 November 2024  11:37:17 +0000 (0:00:00.369)       0:01:13.948 *******
[WARNING]: Could not match supplied host pattern, ignoring:
enable_outward_rabbitmq_True
[WARNING]: Could not match supplied host pattern, ignoring:
outward_rabbitmq_restart
ok: [testbed-node-0]
ok: [testbed-node-1]
ok: [testbed-node-2]
$ osism console --type clush rabbitmq
Enter 'quit' to leave this interactive mode
Working with nodes: testbed-node-[0-2]
clush> docker exec rabbitmq rabbitmqctl list_feature_flags | grep disabled
testbed-node-1: khepri_db       disabled
testbed-node-0: khepri_db       disabled
testbed-node-2: khepri_db       disabled
maliblatt commented 2 weeks ago

Okay, thank you, that makes sense to me. I will discuss in the team am enable the feature flags.