search

ansible-collections / community.rabbitmq

Manage RabbitMQ with Ansible
http://galaxy.ansible.com/community/rabbitmq
Other
31 stars 50 forks source link

Check RabbitMQ version and parse result of commands as JSON #36

Closed tchernomax closed 4 years ago

tchernomax commented 4 years ago

Author: ptzianos

from https://github.com/ansible/ansible/pull/66876

SUMMARY

This PR is a refactoring of the RabbitMQ User module so that in the beginning of every operation it checks the version of the daemon and then, whenever possible, adds flags to the rabbitmqctl so that the output is a JSON document which can then be parsed in an easier and less "hacky" way. It is a more permanent solution relative to this PR. Specifically for versions 3.7 and newer of RabbitMQ, the --formatter flag is used to output JSON, while in versions before that, the -q flag is added to prevent unnecessary output to be printed and reduce parsing errors. The _exec method has been refactored to allow the called to decide whether or not it wants an error to immediately cause the execution to stop, or if the error should be returned and handled on a higher level. This is done to allow the module the try to check for the version of the daemon in multiple different ways. This has been tested with the following RabbitMQ versions:

* 3.6.10

* 3.6.16

* 3.7.6

* 3.7.6

* 3.7.7

* 3.7.8

* 3.7.9

* 3.7.10

* 3.7.11

* 3.7.12

* 3.7.13

* 3.7.14
SUMMARY

This PR is a refactoring of the RabbitMQ User module so that in the beginning of every operation it checks the version of the daemon and then, whenever possible, adds flags to the rabbitmqctl so that the output is a JSON document which can then be parsed in an easier and less "hacky" way. It is a more permanent solution relative to this PR. Specifically for versions 3.7 and newer of RabbitMQ, the --formatter flag is used to output JSON, while in versions before that, the -q flag is added to prevent unnecessary output to be printed and reduce parsing errors. The _exec method has been refactored to allow the called to decide whether or not it wants an error to immediately cause the execution to stop, or if the error should be returned and handled on a higher level. This is done to allow the module the try to check for the version of the daemon in multiple different ways. This has been tested with the following RabbitMQ versions:

* 3.6.10

* 3.6.16

* 3.7.6

* 3.7.6

* 3.7.7

* 3.7.8

* 3.7.9

* 3.7.10

* 3.7.11

* 3.7.12

* 3.7.13

* 3.7.14

* 3.7.15

* 3.7.16

* 3.7.17

* 3.7.18

* 3.7.19

* 3.7.20

* 3.7.21

* 3.7.22

* 3.7.23

* 3.8.0

* 3.8.1

* 3.8.2

All versions where deployed on an Ubuntu 18.04 Vagrant VM using an Ansible script. Samples of the command outputs have been added in the rabbitmq_user_fixtures.py file. Tests have been expanded with output from the daemon process from various versions. Now tests cover both the behavior of the module but also the parsing of input. A small issue has also been fixed, where the module is performing more operations than needed when changing the policies of a vhost. Specifically, if a user's policy is changing, then it was first deleted and then added again. In the current PR this is changed to only a single set operation that overwrites the previous policy.

Fixes #48890, addresses final part of #29281

ISSUE TYPE
* Bugfix Pull Request
COMPONENT NAME

lib/ansible/modules/messaging/rabbitmq/rabbitmq_user.py

ADDITIONAL INFORMATION

In 3.7.* versions of RabbitMQ, a weird issue was discovered, where rabbitmqctl would return unicode codes instead of the actual characters. This was fixed by checking the contents of the output and if they are integers to be converted into the unicode characters. No better way was found to solve this. As mentioned previously, this was tested using Vagrant and two simple Ansible scripts:

all_versions.yml

---
- hosts: all
  vars:
  remote_user: vagrant
  tasks:
  - name: "Test RabbitMQ user module with server {{ item }}"
    include_tasks: rabbitmq.yml
    vars:
      rabbitmq_version: "{{ item.rabbitmq_version }}"
      erlang_verion: "{{ item.erlang_verion }}"
    with_items:
    - {"rabbitmq_version": "3.6.10-1", "erlang_verion": "19.x"}
    - {"rabbitmq_version": "3.6.16-2", "erlang_verion": "20.x"}
    - {"rabbitmq_version": "3.7.6-1", "erlang_verion": "20.x"}
    - {"rabbitmq_version": "3.7.7-1", "erlang_verion": "21.x"}
    - {"rabbitmq_version": "3.7.8-1", "erlang_verion": "21.x"}
    - {"rabbitmq_version": "3.7.9-1", "erlang_verion": "21.x"}
    - {"rabbitmq_version": "3.7.10-1", "erlang_verion": "21.x"}
    - {"rabbitmq_version": "3.7.11-1", "erlang_verion": "21.x"}
    - {"rabbitmq_version": "3.7.12-1", "erlang_verion": "21.x"}
    - {"rabbitmq_version": "3.7.13-1", "erlang_verion": "21.x"}
    - {"rabbitmq_version": "3.7.14-1", "erlang_verion": "21.x"}
    - {"rabbitmq_version": "3.7.15-1", "erlang_verion": "22.x"}
    - {"rabbitmq_version": "3.7.16-1", "erlang_verion": "22.x"}
    - {"rabbitmq_version": "3.7.17-1", "erlang_verion": "22.x"}
    - {"rabbitmq_version": "3.7.18-1", "erlang_verion": "22.x"}
    - {"rabbitmq_version": "3.7.19-1", "erlang_verion": "22.x"}
    - {"rabbitmq_version": "3.7.20-1", "erlang_verion": "22.x"}
    - {"rabbitmq_version": "3.7.21-1", "erlang_verion": "22.x"}
    - {"rabbitmq_version": "3.7.22-1", "erlang_verion": "22.x"}
    - {"rabbitmq_version": "3.7.23-1", "erlang_verion": "22.x"}
    - {"rabbitmq_version": "3.8.0-1", "erlang_verion": "22.x"}
    - {"rabbitmq_version": "3.8.1-1", "erlang_verion": "22.x"}
    - {"rabbitmq_version": "3.8.2-1", "erlang_verion": "22.x"}

rabbitmq.yml

---
- name: Add RabbitMQ keyserver
  apt_key:
    keyserver: hkps.pool.sks-keyservers.net
    id: "0x6B73A36E6026DFCA"
  become: yes
  become_user: root
  become_method: sudo
- name: Add trusted key of RabbitMQ repo
  apt_key:
    url: "https://github.com/rabbitmq/signing-keys/releases/download/2.0/rabbitmq-release-signing-key.asc"
    state: present
  become: yes
  become_user: root
  become_method: sudo
- name: Cleanup Erlang Debian repo
  file:
    path: /etc/apt/sources.list.d/erlang.list
    state: absent
  become: yes
  become_user: root
  become_method: sudo
- name: Add Erlang Debian repo
  apt_repository:
    repo: "deb https://dl.bintray.com/rabbitmq-erlang/debian bionic erlang-{{ erlang_verion }}"
    state: present
    filename: erlang
  become: yes
  become_user: root
  become_method: sudo
- name: Add RabbitMQ Debian repo
  apt_repository:
    repo: deb https://dl.bintray.com/rabbitmq/debian bionic main
    state: present
    filename: rabbitmq
  become: yes
  become_user: root
  become_method: sudo
- name: "Install RabbitMQ {{ rabbitmq_version }}"
  apt:
    name: "rabbitmq-server={{ rabbitmq_version }}"
    update_cache: yes
  become: yes
  become_user: root
  become_method: sudo
- name: Register RabbitMQ admins as root. Might cause changes
  rabbitmq_user:
    user: "guest"
    password: "guest2"
    state: present
    tags: administrator,management
    update_password: 'always'
  become: yes
  become_user: root
  become_method: sudo
- name: Register RabbitMQ admins as rabbitmq. No changes detected
  rabbitmq_user:
    user: "guest"
    password: "guest2"
    state: present
    tags: administrator,management
    update_password: 'always'
  become: yes
  become_user: rabbitmq
  become_method: sudo
- name: Register RabbitMQ admins as root. No changes detected
  rabbitmq_user:
    user: "guest"
    password: "guest2"
    state: present
    tags: administrator, management
    update_password: 'always'
  become: yes
  become_user: root
  become_method: sudo
- name: Change RabbitMQ admin tags. Changes detected
  rabbitmq_user:
    user: "guest"
    password: "guest2"
    state: present
    tags: administrator, management, bla
    update_password: 'always'
  become: yes
  become_user: root
  become_method: sudo
- name: Change RabbitMQ admin tags. Changes detected
  rabbitmq_user:
    user: "guest"
    password: "guest2"
    state: present
    tags: administrator, management
    update_password: 'always'
  become: yes
  become_user: root
  become_method: sudo
- name: Register RabbitMQ admins as vagrant which will fail
  rabbitmq_user:
    user: "guest"
    password: "guest2"
    state: present
    tags: administrator,management
    update_password: 'always'
  ignore_errors: yes
- name: Add a dummy user. Changes detected
  rabbitmq_user:
    user: "guest2"
    password: "guest2"
    state: present
  become: yes
  become_user: root
  become_method: sudo
- name: Delete dummy user. Changes detected
  rabbitmq_user:
    user: "guest2"
    state: absent
  become: yes
  become_user: root
  become_method: sudo
- name: Delete dummy user. No changes detected
  rabbitmq_user:
    user: "guest2"
    state: absent
  become: yes
  become_user: root
  become_method: sudo
- name: Ensure test vhost exists
  rabbitmq_vhost:
    name: /test
    state: present
  become: yes
  become_user: root
  become_method: sudo
- name: Fix dummy permissions to guest user. Changes detected
  rabbitmq_user:
    user: "guest"
    state: present
    tags: administrator,management
    permissions:
    - vhost: /
      configure_priv: ^$
      read_priv: ^$
      write_priv: ^$
    - vhost: /test
      configure_priv: ^$
      read_priv: ^pavlos-.*
      write_priv: ^$
  become: yes
  become_user: root
  become_method: sudo
- name: Add dummy permissions to guest user for test vhost. No changes detected
  rabbitmq_user:
    user: "guest"
    state: present
    tags: administrator,management
    vhost: /test
    configure_priv: ^$
    read_priv: ^pavlos-.*
    write_priv: ^$
  become: yes
  become_user: root
  become_method: sudo
- name: Fix dummy permissions to guest user. Changes detected
  rabbitmq_user:
    user: "guest"
    state: present
    tags: administrator,management
    permissions:
    - vhost: /
      configure_priv: ^$
      read_priv: ^$
      write_priv: ^$
    - vhost: /test
      configure_priv: ^$
      read_priv: ^$
      write_priv: ^$
  become: yes
  become_user: root
  become_method: sudo
mjrider commented 4 years ago

just tested this with ansible 2.9.10 and rabbitmq 3.7.26-1 and this fixes the issue for me