[BUG] Defining saltenv options in gitfs_remotes are not rendered correctly.

[bennodepenno]

Your setup

Formula commit hash / release tag

f10a85fc706aeba06453234b02bb5d8b0f00ad34

Versions reports (master & minion)

Salt Version: Salt: 3000.3

Dependency Versions: cffi: 1.6.0 cherrypy: Not Installed dateutil: Not Installed docker-py: Not Installed gitdb: Not Installed gitpython: Not Installed Jinja2: 2.7.2 libgit2: 0.26.3 M2Crypto: Not Installed Mako: Not Installed msgpack-pure: Not Installed msgpack-python: 0.6.2 mysql-python: Not Installed pycparser: 2.14 pycrypto: 2.6.1 pycryptodome: Not Installed pygit2: 0.26.4 Python: 2.7.5 (default, Apr 2 2020, 13:16:51) python-gnupg: Not Installed PyYAML: 3.11 PyZMQ: 15.3.0 smmap: Not Installed timelib: Not Installed Tornado: 4.5.3 ZMQ: 4.1.4

System Versions: dist: centos 7.8.2003 Core locale: ANSI_X3.4-1968 machine: x86_64 release: 3.10.0-1127.8.2.el7.x86_64 system: Linux version: CentOS Linux 7.8.2003 Core

Pillar / config used

salt:
  release: '3000.3'
  master_remove_config: True
  master:
    interface: 0.0.0.0
    worker_threads: 15
    enable_gpu_grains: True
    autosign_file: /etc/salt/autosign.conf
    minion_data_cache: True
    fileserver_backend:
      - git
      - roots
    file_roots:
      base:
        - /srv/uv/salt
        - /srv/uv/formulas
    pillar_roots:
      base:
        - /srv/uv/pillar
    ext_pillar:
      - git:
        - develop https://myurl/pillar.git:
          - password: 'secret'
          - user: 'saltuser'
          - env: uv
          - root: uv
    gitfs_saltenv_whitelist:
      - base
    gitfs_update_interval: 60
    gitfs_provider: pygit2
    gitfs_insecure_auth: True
    git_pillar_insecure_auth: True
    file_client: local
    gitfs_remotes:
      - https://myurl/salt.git:
        - user: 'saltuser'
        - password: 'secret'
        - mountpoint: salt://
        - root: uv
        - saltenv:
          - uv:
            - ref: develop
      - https://myurl/formulas/bind-formula.git:
        - user: 'saltuser'
        - password: 'secret'
        - root: bind
        - saltenv:
          - uv:
            - ref: master
            - mountpoint: salt://bind
      - https://myurl/formulas/dhcpd-formula.git:
        - user: 'saltuser'
        - password: 'secret'
        - root: dhcpd
        - saltenv:
          - uv:
            - ref: master
            - mountpoint: salt://dhcpd
      - https://myurl/formulas/salt-formula.git:
        - user: 'saltuser'
        - password: 'secret'
        - root: salt
        - saltenv:
          - uv:
            - ref: master
            - mountpoint: salt://salt
      - https://myurl/formulas/users-formula.git:
        - user: 'saltuser'
        - password: 'secret'
        - root: users
        - saltenv:
          - uv:
            - ref: master
            - mountpoint: salt://users
      - https://myurl/formulas/sudoers-formula.git:
        - user: 'saltuser'
        - password: 'secret'
        - root: sudoers
        - saltenv:
          - uv:
            - ref: master
            - mountpoint: salt://sudoers
      - https://myurl/formulas/grafana-formula.git:
        - user: 'saltuser'
        - password: 'secret'
        - root: grafana
        - saltenv:
          - uv:
            - ref: master
            - mountpoint: salt://grafana
      - https://myurl/formulas/rsyslog-formula.git:
        - user: 'saltuser'
        - password: 'secret'
        - root: rsyslog
        - saltenv:
          - uv:
            - ref: master
            - mountpoint: salt://rsyslog
      - https://myurl/formulas/prometheus-formula.git:
        - user: 'saltuser'
        - password: 'secret'
        - root: prometheus
        - saltenv:
          - uv:
            - ref: master
            - mountpoint: salt://prometheus
    decrypt_pillar:
      - 'certificates:vault'
    decrypt_pillar_delimiter: '|'
    use_superseded:
      - module.run
    external_auth:
      pam:
        saltuser:
          - '@runner'
        foreman:
          - '@runner'
          - '@wheel'
        foreman-proxy:
          - '@runner'
          - '@wheel'

    reactors:
      - 'master/deploy':
        - /srv/salt/reactors/deploy.sls

---

Bug details

Describe the bug

Hi,

I'm having issues using the salt-formula when using gitfs_remotes. In my pillar, I've specified:

gitfs_remotes:
      - https://myurl/formulas/bind-formula.git:
        - user: 'saltuser@myurl'
        - password: 'secret'
        - root: bind
        - saltenv:
          - uv:
            - ref: master
            - mountpoint: salt://bind

When rendered, the output shows:

gitfs_remotes:
    - https://myurl/formulas/bind-formula.git:
      - user: saltuser
      - password: secret
      - root: bind
      - saltenv: [{u'uv': [{u'ref': u'master'}, {u'mountpoint': u'salt://bind'}]}]

It's this line which is incorrect: [{u'uv': [{u'ref': u'master'}, {u'mountpoint': u'salt://bind'}]}].

Steps to reproduce the bug

Use the pillar set and execute the salt formula on target host. The salt run will succeed but the resulting file /etc/salt/master.d/f_defaults.conf contains the incorrect configuration.

Expected behaviour

gitfs_remotes:
    - https://myurl/formulas/bind-formula.git:
      - user: saltuser
      - password: secret
      - root: bind
      - saltenv:
         - uv:
           - ref: master
           - mountpoint: salt://bind

Attempts to fix the bug

I've attempted some suggestions to use "{ -uv: }" and "{[ -uv ]}" without luck.

Additional context

[myii]

@bennodepenno Thanks for the report. This is known problem with the pillar approach, so we're in the process of moving to providing master/minion configuration files via. TOFS instead, which was implemented in #398 (specifically starting from https://github.com/saltstack-formulas/salt-formula/pull/398#issuecomment-499354351). It's working but there are steps remaining to make it easier for users to adopt (#417). An example of helping someone getting it working has been captured around here:

• https://freenode.logbot.info/saltstack-formulas/20200320#c3444154

Hopefully, there's enough there to help you work out how to provide the configuration files via. TOFS. We were hoping for some documentation to be provided but that's not the case so far. If you need further help, feel free to start a conversation in the #formulas channel in Slack (also available via. IRC).

[bennodepenno]

Thank you for your reply. I will look into TOFS and the examples provided. Thank you for your time and effort. I would like to participate on Slack, but I do not know how to join the #formulas channel.

[myii]

@bennodepenno This should be the correct link, please let me know if it doesn't work:

3. Please direct questions to the #formulas channel on Slack, which is bridged to #saltstack-formulas on Freenode.

[bennodepenno]

Thanks for all the help.