mesaguy / ansible-prometheus

Ansible role for the management of Prometheus software and Prometheus exporters
https://galaxy.ansible.com/mesaguy/prometheus
MIT License
71 stars 31 forks source link

Using local rules file #37

Closed coderfool closed 2 years ago

coderfool commented 3 years ago

I have a custom config file for Prometheus server in which 'rule_files' points to a file on that server. This rule file has to be copied from the controller. In prometheus.yml, the prometheus server config file is created before copying the local rule files:

- name: Setup {{ prometheus_software_name_version }} configuration file
  become: true
  template:
    src: 'templates/generic-config.yml.j2'
    dest: '{{ prometheus_etc_dir }}/prometheus.yml'
    owner: root
    group: '{{ prometheus_group }}'
    mode: 0640
    backup: true
    validate: '{{ prometheus_software_install_dir }}/promtool check config %s'
  notify:
    - Reload Prometheus service

- name: Include task to setup Prometheus rules
  include_tasks: _rules.yml
  when: prometheus_manage_rules | default(false) | bool

This runs into a validation error because the rule file doesn't exist yet:

fatal: [prometheus]: FAILED! => {"changed": false, "checksum": "de45e00cd186d6c11d37e4ce5eebe6fe1d00eb27", "exit_status": 1, "msg": "failed to validate", "stderr": "  FAILED: \"/apps/shared/monitoring/prometheus/rules.yml\" does not point to an existing file\n", "stderr_lines": ["  FAILED: \"/apps/shared/monitoring/prometheus/rules.yml\" does not point to an existing file"], "stdout": "Checking /root/.ansible/tmp/ansible-tmp-1632221852.5072598-70971-236443730807695/source\n\n", "stdout_lines": ["Checking /root/.ansible/tmp/ansible-tmp-1632221852.5072598-70971-236443730807695/source", ""]}

It works fine when I switch the order of these 2 tasks.

Playbook:

- connection: docker
  hosts: prometheus
  vars:
    prometheus_components:
      - prometheus
    prometheus_version: v2.27.1
    prometheus_root_dir: /apps/shared/monitoring/prometheus
    prometheus_manage_rules: true
    prometheus_rules_dir: '{{ prometheus_root_dir }}'
    prometheus_rules_source_dirs:
      - './rules'
    prometheus_server_cfg: '{{ lookup("file", "./prometheus-config.yml") | from_yaml }}'
  roles:
    - mesaguy.prometheus

Am I missing something?

mesaguy commented 3 years ago

If I am reading the situtation correctly, maybe your Prometheus configuration file references/includes a file called "rules.yml"?

When the Prometheus configuration file is created, the 'promtool' configuration validation runs, the referenced "rules.yml" file doesn't yet exist, and an error is raised by promtool because of the missing rules file.

coderfool commented 3 years ago

If I am reading the situtation correctly, maybe your Prometheus configuration file references/includes a file called "rules.yml"?

When the Prometheus configuration file is created, the 'promtool' configuration validation runs, the referenced "rules.yml" file doesn't yet exist, and an error is raised by promtool because of the missing rules file.

That is correct. I want to know how to deploy the rules.yml from the Ansible controller to the Prometheus server.

@mesaguy