search

saltstack-formulas / mysql-formula

Install the MySQL client and/or server
http://docs.saltstack.com/en/latest/topics/development/conventions/formulas.html
Other
85 stars 367 forks source link

[BUG] Setting mountpoint breaks tplroot #245

Open taranlu-houzz opened 4 years ago

taranlu-houzz commented 4 years ago

Your setup

Formula commit hash / release tag

Using default, so I would assume that would be either master or base.

Versions reports (master & minion)

master:

Salt Version:
           Salt: 3000.1

Dependency Versions:
           cffi: 1.12.2
       cherrypy: unknown
       dateutil: 2.8.0
      docker-py: Not Installed
          gitdb: 2.0.6
      gitpython: 2.1.15
         Jinja2: 2.10.1
        libgit2: 1.0.0
       M2Crypto: 0.35.2
           Mako: 1.0.7
   msgpack-pure: Not Installed
 msgpack-python: 0.5.6
   mysql-python: Not Installed
      pycparser: 2.19
       pycrypto: 3.8.1
   pycryptodome: Not Installed
         pygit2: 1.2.0
         Python: 3.7.7 (default, Mar 10 2020, 15:43:27)
   python-gnupg: 0.4.4
         PyYAML: 5.1.2
          PyZMQ: 18.0.1
          smmap: 3.0.2
        timelib: 0.2.4
        Tornado: 4.5.3
            ZMQ: 4.3.1

System Versions:
           dist:
         locale: UTF-8
        machine: x86_64
        release: 17.7.0
         system: Darwin
        version: 10.13.6 x86_64

minion:

Salt Version:
           Salt: 3000.3

Dependency Versions:
           cffi: 1.12.2
       cherrypy: unknown
       dateutil: 2.8.0
      docker-py: Not Installed
          gitdb: 2.0.6
      gitpython: 2.1.15
         Jinja2: 2.10.1
        libgit2: 1.0.0
       M2Crypto: 0.35.2
           Mako: 1.0.7
   msgpack-pure: Not Installed
 msgpack-python: 0.5.6
   mysql-python: Not Installed
      pycparser: 2.19
       pycrypto: 3.8.1
   pycryptodome: Not Installed
         pygit2: 1.2.1
         Python: 3.7.7 (default, Mar 10 2020, 15:43:27)
   python-gnupg: 0.4.4
         PyYAML: 5.1.2
          PyZMQ: 18.0.1
          smmap: 3.0.4
        timelib: 0.2.4
        Tornado: 4.5.3
            ZMQ: 4.3.1

System Versions:
           dist:
         locale: UTF-8
        machine: x86_64
        release: 17.7.0
         system: Darwin
        version: 10.13.6 x86_64

Pillar / config used

No pillar config.

Bug details

Describe the bug

Adding a different mountpoint (e.g. salt://external) for the gitfs seems to break things:

----@---- /e/salt> sudo salt '----' state.apply external.mysql.macos.remove test=True
----:
    Data failed to compile:
----------
    Rendering SLS 'base:external.mysql.macos.remove' failed: Jinja error: external/defaults.yaml
/var/cache/salt/minion/files/base/external/mysql/map.jinja(7):
---
[...]
# vim: ft=jinja

{#- Get the `tplroot` from `tpldir` #}
{%- set tplroot = tpldir.split('/')[0] %}
{#- Start imports as #}
{%- import_yaml tplroot ~ "/defaults.yaml" as default_settings %}    <======================
{%- import_yaml tplroot ~ "/osfamilymap.yaml" as osfamilymap %}
{%- import_yaml tplroot ~ "/osmap.yaml" as osmap %}
{%- import_yaml tplroot ~ "/osfingermap.yaml" as osfingermap %}

{#- Retrieve the config dict only once #}
[...]
---
Traceback (most recent call last):
  File "/usr/local/Cellar/salt/3000.3/libexec/lib/python3.7/site-packages/salt/utils/templates.py", line 394, in render_jinja_tmpl
    output = template.render(**decoded_context)
  File "/usr/local/Cellar/salt/3000.3/libexec/lib/python3.7/site-packages/jinja2/asyncsupport.py", line 76, in render
    return original_render(self, *args, **kwargs)
  File "/usr/local/Cellar/salt/3000.3/libexec/lib/python3.7/site-packages/jinja2/environment.py", line 1008, in render
    return self.environment.handle_exception(exc_info, True)
  File "/usr/local/Cellar/salt/3000.3/libexec/lib/python3.7/site-packages/jinja2/environment.py", line 780, in handle_exception
    reraise(exc_type, exc_value, tb)
  File "/usr/local/Cellar/salt/3000.3/libexec/lib/python3.7/site-packages/jinja2/_compat.py", line 37, in reraise
    raise value.with_traceback(tb)
  File "<template>", line 4, in top-level template code
  File "/usr/local/Cellar/salt/3000.3/libexec/lib/python3.7/site-packages/jinja2/environment.py", line 1073, in make_module
    return TemplateModule(self, self.new_context(vars, shared, locals))
  File "/usr/local/Cellar/salt/3000.3/libexec/lib/python3.7/site-packages/jinja2/environment.py", line 1152, in __init__
    body_stream = list(template.root_render_func(context))
  File "/var/cache/salt/minion/files/base/external/mysql/map.jinja", line 7, in top-level template code
    {%- import_yaml tplroot ~ "/defaults.yaml" as default_settings %}
  File "/usr/local/Cellar/salt/3000.3/libexec/lib/python3.7/site-packages/salt/utils/jinja.py", line 193, in get_source
    raise TemplateNotFound(template)
jinja2.exceptions.TemplateNotFound: external/defaults.yaml

It looks like it is mistakenly trying to look for external/defaults.yaml when it should be something more like external/mysql/defaults.yaml or maybe external/mysql-formula/mysql/defaults.yaml.

Steps to reproduce the bug

Add the following entry to /etc/salt/master config:

gitfs_remotes:
  - https://github.com/saltstack-formulas/mysql-formula.git:
    - mountpoint: salt://external

Expected behaviour

I would expect the formula to be able to handle having a different mountpoint in the file server.

gavriluk commented 8 months ago

Try to use root parameter https://docs.saltproject.io/en/latest/ref/configuration/master.html#std-conf_master-gitfs_root

gitfs_remotes:
  - https://github.com/saltstack-formulas/mysql-formula.git:
    - mountpoint: salt://external
    - root: mysql
baby-gnu commented 8 months ago

This is a known issue since a long time.

The proper solution would be to use salt.modules.slsutil.findup but it has a major issue for salt-ssh and is waiting 3007.