vitabaks / postgresql_cluster

PostgreSQL High-Availability Cluster (based on Patroni). Automating with Ansible.
https://postgresql-cluster.org
MIT License
1.69k stars 410 forks source link

update_pgcluster.yml: Reboot node if it's required, e.g. kernel or security updates #510

Closed chuegel closed 11 months ago

chuegel commented 11 months ago

Environments like PCIDSS require to apply kernel and security updates on a regular basis. Once those updates have been applied, a reboot of the host is needed. This PR will reboot a host only if it's required (kernel updates, security updates etc.) when updating the PostgreSQL cluster with ansible-playbook update_pgcluster.yml -e target=system

chuegel commented 11 months ago

The failing tests are expected as the docker containers die on reboot. Not sure how to catch this test. Vagrant?

What worries me is, as I mentioned in the discussion the handler get called multiple times when notified. This would need a thorough testing.

vitabaks commented 11 months ago

Thank you @chuegel for your contribution!

The failing tests are expected as the docker containers die on reboot.

You can exclude this task from containers

ansible_virtualization_type not in ['container', 'docker', 'lxc', 'podman']  # exclude for containers to prevent test failures in CI.
vitabaks commented 11 months ago

What worries me is, as I mentioned in the discussion the handler get called multiple times when notified.

perhaps it is necessary to move from "handler" to a simple task.

vitabaks commented 11 months ago

perhaps it is necessary to move from "handler" to a simple task.

Example:

- name: Check if a reboot is required
  ansible.builtin.stat:
    path: /var/run/reboot-required
  register: reboot_required_debian
  changed_when: false
  when:
    - ansible_os_family == "Debian"
    - ansible_virtualization_type not in ['container', 'docker', 'lxc', 'podman']  # exclude for containers to prevent test failures in CI.

- name: Check if a reboot is required
  ansible.builtin.command: needs-restarting -r
  register: reboot_required_rhel
  failed_when: false
  changed_when: false
  when:
    - ansible_os_family == "RedHat"
    - ansible_virtualization_type not in ['container', 'docker', 'lxc', 'podman']  # exclude for containers to prevent test failures in CI.

- name: Rebooting host
  ansible.builtin.reboot:
    msg: "Reboot initiated by Ansible due to required system updates"
    reboot_timeout: 1800 # 30 minutes
    test_command: uptime
  when: (reboot_required_debian.stat.exists is defined and reboot_required_debian.stat.exists) or
        (reboot_required_rhel.rc is defined and reboot_required_rhel.rc != 0)
chuegel commented 11 months ago

perhaps it is necessary to move from "handler" to a simple task.

Example:

- name: Check if a reboot is required
  ansible.builtin.stat:
    path: /var/run/reboot-required
  register: reboot_required_debian
  changed_when: false
  when:
    - ansible_os_family == "Debian"
    - ansible_virtualization_type not in ['container', 'docker', 'lxc', 'podman']  # exclude for containers to prevent test failures in CI.

- name: Check if a reboot is required
  ansible.builtin.command: needs-restarting -r
  register: reboot_required_rhel
  failed_when: false
  changed_when: false
  when:
    - ansible_os_family == "RedHat"
    - ansible_virtualization_type not in ['container', 'docker', 'lxc', 'podman']  # exclude for containers to prevent test failures in CI.

- name: Rebooting host
  ansible.builtin.reboot:
    msg: "Reboot initiated by Ansible due to required system updates"
    reboot_timeout: 1800 # 30 minutes
    test_command: uptime
  when: (reboot_required_debian.stat.exists is defined and reboot_required_debian.stat.exists) or
        (reboot_required_rhel.rc is defined and reboot_required_rhel.rc != 0)

Yeah, that looks good, thank you. I tested it with an 3 node Ubuntu 22.04 cluster and it works as expected.