saltstack-formulas / prometheus-formula

Manage a Prometheus installation
Other
27 stars 51 forks source link

[BUG] service file lookup uses {{ name }} as raw string #24

Closed viq closed 4 years ago

viq commented 4 years ago

Your setup

Formula commit hash / release tag

d9c93860385303ae89025431da7a83d48c5a6adf

Versions reports (master & minion)

salt-call --versions-report
Salt Version:
           Salt: 2019.2.2

Dependency Versions:
           cffi: 1.5.2
       cherrypy: Not Installed
       dateutil: 2.4.2
      docker-py: 1.9.0
          gitdb: 0.6.4
      gitpython: 1.0.1
          ioflo: Not Installed
         Jinja2: 2.8
        libgit2: 0.24.0
        libnacl: Not Installed
       M2Crypto: Not Installed
           Mako: Not Installed
   msgpack-pure: Not Installed
 msgpack-python: 0.4.6
   mysql-python: Not Installed
      pycparser: 2.14
       pycrypto: 2.6.1
   pycryptodome: Not Installed
         pygit2: 0.24.0
         Python: 3.5.2 (default, Oct  8 2019, 13:06:37)
   python-gnupg: 0.3.8
         PyYAML: 3.11
          PyZMQ: 15.2.0
           RAET: Not Installed
          smmap: 0.9.0
        timelib: Not Installed
        Tornado: 4.2.1
            ZMQ: 4.1.4

System Versions:
           dist: Ubuntu 16.04 xenial
         locale: UTF-8
        machine: x86_64
        release: 4.4.0-1092-aws
         system: Linux
        version: Ubuntu 16.04 xenial

Pillar / config used

prometheus:
  tofs:
    source_files:
      {% raw %}prometheus-archive-install-{{ name }}-managed-service:{% endraw %}
        - node_exporter.service.jinja
  wanted:
    - prometheus
    - node_exporter
    - alertmanager
  use_upstream_archive: True
  pkg:
    prometheus:
      archive_version: '2.14.0'
      archive_hash: ddf47223ec716594254df110ee26ad52e66d7d4471ae8edbf9020087396861e5
    node_exporter:
      archive_version: '0.18.1'
      archive_hash: b2503fd932f85f4e5baf161268854bf5d22001869b84f00fd2d1f57b51b72424
      name: node_exporter
      args_file: /etc/default/node_exporter
    alertmanager:
      archive_version: '0.19.0'
      archive_hash: 6191788f8b91a05955fcdc2ab4bc7d0edf70e5039f3acffb284d75745cc80d67
  service:
    prometheus:
      args:
        web.listen-address: "127.0.0.1:9090"
    node_exporter:
      args:
        web.listen-address: "127.0.0.1:9100"
  config:
    prometheus:
      global:
        scrape_interval: 10s
        evaluation_interval: 10s
      alerting:
        alertmanagers:
          - static_configs:
            - targets:
              - localhost:9093
      scrape_configs:
        - job_name: 'prometheus'
          static_configs:
            - targets: ['localhost:9090']
        - job_name: 'node'
          static_configs:
            - targets: ['localhost:9100']

where the most relevant part is

prometheus:
  tofs:
    source_files:
      {% raw %}prometheus-archive-install-{{ name }}-managed-service:{% endraw %}
        - node_exporter.service.jinja

Bug details

Describe the bug

According to all my reading, and most specifically https://github.com/saltstack-formulas/prometheus-formula/blob/master/prometheus/archive/install.sls#L61-L62 pillar like

prometheus:
  tofs:
    source_files:
      {% raw %}prometheus-archive-install-{{ name }}-managed-service:{% endraw %}
        - node_exporter.service.jinja

should result in state managing node_exporter service to look for salt://prometheus/files/default/node_exporter.service.jinja, yet it didn't. I saw in logs this:

[DEBUG   ] In saltenv 'base', looking at rel_path 'prometheus/libtofs.jinja' to resolve 'salt://prometheus/libtofs.jinja'
[DEBUG   ] In saltenv 'base', ** considering ** path '/var/cache/salt/minion/files/base/prometheus/libtofs.jinja' to resolve 'salt://prometheus/libtofs.jinja'
[DEBUG   ] LazyLoaded config.get
[DEBUG   ] key: prometheus:tofs:path_prefix, ret: _|-
[DEBUG   ] key: prometheus:tofs:dirs:files, ret: _|-
[DEBUG   ] key: prometheus:tofs:files_switch, ret: _|-
[DEBUG   ] key: prometheus:tofs:files:prometheus-archive-install-{{ name }}-managed-service, ret: _|-
[DEBUG   ] key: prometheus:files_switch, ret: _|-
[DEBUG   ] key: prometheus:tofs:dirs:default, ret: _|-
[DEBUG   ] key: prometheus:tofs:dirs:default, ret: _|-
[DEBUG   ] key: prometheus:tofs:path_prefix, ret: _|-
[DEBUG   ] key: prometheus:tofs:dirs:files, ret: _|-
[DEBUG   ] key: prometheus:tofs:files_switch, ret: _|-
[DEBUG   ] key: prometheus:tofs:files:prometheus-archive-install-{{ name }}-managed-service, ret: _|-
[DEBUG   ] key: prometheus:files_switch, ret: _|-
[DEBUG   ] key: prometheus:tofs:dirs:default, ret: _|-
[DEBUG   ] key: prometheus:tofs:dirs:default, ret: _|-
[DEBUG   ] key: prometheus:tofs:path_prefix, ret: _|-
[DEBUG   ] key: prometheus:tofs:dirs:files, ret: _|-
[DEBUG   ] key: prometheus:tofs:files_switch, ret: _|-
[DEBUG   ] key: prometheus:tofs:files:prometheus-archive-install-{{ name }}-managed-service, ret: _|-
[DEBUG   ] key: prometheus:files_switch, ret: _|-
[DEBUG   ] key: prometheus:tofs:dirs:default, ret: _|-
[DEBUG   ] key: prometheus:tofs:dirs:default, ret: _|-

and the presence multiple times of {{ name }} was suspicious. So I made an experiment, and set pillar like so:

prometheus:
  tofs:
    source_files:
      {% raw %}prometheus-archive-install-{{ name }}-managed-service:{% endraw %}
        - node_exporter.service.jinja

resulting in

salt-call pillar.get prometheus:tofs:source_files
local:
    ----------
    prometheus-archive-install-{{ name }}-managed-service:
        - node_exporter.service.jinja

Suddenly all the states managing services started looking for salt://prometheus/files/default/node_exporter.service.jinja

Steps to reproduce the bug

See above.

Expected behaviour

{{ name }} getting properly expanded to current service name.

Attempts to fix the bug

None yet, I don't know enough about jinja templating and where it applies and where it doesn't.

Additional context

Found while trying to figure out https://github.com/saltstack-formulas/prometheus-formula/issues/23

noelmcloughlin commented 4 years ago

30