Open fgionghi opened 2 years ago
fgionghi
commented
2 years ago We tried without chocolatey, so using directly winrepo. My pillar now is:
packages:
pkgs:
wanted:
- prometheus-wmi-exporterbut the problem persist. Pay attention to delete all the cached file on target machine.
myii
commented
2 years ago @fgionghi This is a tough one to reproduce. I used the pillar you provided, with a slight addition:
packages:
chocolatey:
+ required:
+ pkgs: []
wanted:
prometheus-windows-exporter.install:
version: '0.16.0'Tested this in the Windows CI we have for this formula and all of them are successfully installing prometheus-wmi-exporter. The failures shown are actually because I didn't also update the packages that the tests are looking for:
I also raised this issue in the last Formulas working group meeting but no-one had faced this problem, even though they are working with Windows minions.
I reckon the best way to move forward would be to ask a question in the #windows channel in SaltStack's Slack instance. From there, we can hopefully work out where to look next, or if this is actually a bug with Salt itself. Because it's not really specific to this formula, after all.
Let me know how you intend to proceed.
myii
commented
2 years ago @fgionghi I've gone ahead and added a comment in the #windows channel:
Other useful links that might come in handy when attempting to resolve this:
fgionghi
commented
2 years ago We did a test formula called "winpackages":
init.sls:
{% from './map.jinja' import packages with context %}
{% for pkg in packages %}
Choco_package_{{ pkg|upper }}_install:
chocolatey.installed:
- name: {{ pkg }}
{% endfor %}map.jinja:
{% set default_packages = salt['grains.filter_by']({
'Windows': {
'pkgs': [
'googlechrome',
]
},
}) %}
{% set packages = salt['pillar.get']('winpackages:pkgs', default=default_packages.pkgs, merge=True) %}pillar.sls
winpackages:
pkgs:
- prometheus-windows-exporter.installand this is not working, always giving me ./map.jinja not found.
Then we modify the init.sls like this:
init.sls:
{% from 'winpackages/map.jinja' import packages with context %}
{% for pkg in packages %}
Choco_package_{{ pkg|upper }}_install:
chocolatey.installed:
- name: {{ pkg }}
{% endfor %}and this is working.
Our theory is: salt minion is searching ./map.jinja not inside the formula folder, but on the parent folder (C:\salt\var\cache\salt\minion\files\base).
BUT, we did another test:
winpackages/map.jinja)C:\salt\var\cache\salt\minion\files\base\winpackages we have both map.jinja and init.sls) ./map.jinja) is now working.
So our theory is wrong: the salt-minion is not searching in the parent formula (at least when the formula's files are cached). Could it be that we have some config in the salt-master that broke this relative path thing (only on windows minions)?
The config of the salt-minion is very simple: master: our-salt-master.our-domain.com
Our test will fail only if C:\salt\var\cache\salt\minion\files\base is empty. We use salt skyros.pc.itc.it saltutil.clear_cache.
Sorry if we are not precise, if you need further information let me know.
mvivaldi
commented
2 years ago DISCLAIMER: I am a colleague of @fgionghi I tried in my installation at home the "winpackages" from the last post and I have the same error. But we are using ubuntu 20.04 in all of our tests, could it be something related to the distro? I tried to downgrade the salt-master to the 3001.8+ds-1 version, but nothing changed. Maybe we are doing something wrong and we can't see it, someone could try to do the same thing with an ubuntu 20.04 master and verify if the problem is here? Thank you
myii
commented
2 years ago Thanks for the further info, @mvivaldi. I've now been able to reproduce this bug by connecting the Windows 10 VirtualBox VM that we use for testing this formula to my Salt master (on Ubuntu 20.04):
Traceback:
DESKTOP-MG2P5NV:
Data failed to compile:
----------
Rendering SLS 'base:packages.chocolatey' failed: Jinja error: ./defaults.yaml
c:\salt\var\cache\salt\minion\files\base\packages\map.jinja(4):
---
# -*- coding: utf-8 -*-
# vim: ft=jinja
{%- import_yaml './defaults.yaml' as defaults %} <======================
{%- import_yaml './osfamilymap.yaml' as osfamilymap %}
{%- import_yaml './osmap.yaml' as osmap %}
{%- import_yaml './osfingermap.yaml' as osfingermap %}
{%- set packages = salt['grains.filter_by'](
[...]
---
Traceback (most recent call last):
File "c:\salt\bin\lib\site-packages\salt-3003-py3.7.egg\salt\utils\templates.py", line 497, in render_jinja_tmpl
output = template.render(**decoded_context)
File "c:\salt\bin\lib\site-packages\jinja2\asyncsupport.py", line 76, in render
return original_render(self, *args, **kwargs)
File "c:\salt\bin\lib\site-packages\jinja2\environment.py", line 1008, in render
return self.environment.handle_exception(exc_info, True)
File "c:\salt\bin\lib\site-packages\jinja2\environment.py", line 780, in handle_exception
reraise(exc_type, exc_value, tb)
File "c:\salt\bin\lib\site-packages\jinja2\_compat.py", line 37, in reraise
raise value.with_traceback(tb)
File "<template>", line 3, in top-level template code
File "c:\salt\bin\lib\site-packages\jinja2\environment.py", line 1073, in make_module
return TemplateModule(self, self.new_context(vars, shared, locals))
File "c:\salt\bin\lib\site-packages\jinja2\environment.py", line 1152, in __init__
body_stream = list(template.root_render_func(context))
File "c:\salt\var\cache\salt\minion\files\base\packages\map.jinja", line 4, in top-level template code
{%- import_yaml './defaults.yaml' as defaults %}
File "c:\salt\bin\lib\site-packages\salt-3003-py3.7.egg\salt\utils\jinja.py", line 198, in get_source
raise TemplateNotFound(template)
jinja2.exceptions.TemplateNotFound: ./defaults.yaml
ERROR: Minions returned with non-zero exit codeWhen I get a bit more time, I will test this on Windows 8 as well.
Ultimately, this appears to be a bug in Salt itself, so we'll need to report this to them with all the info we can figure out. It would be useful to test with a Salt master other than Ubuntu 20.04 as well.
myii
commented
2 years ago Steps to reproduce:
$ KITCHEN_LOCAL_YAML=kitchen.vagrant.yml bin/kitchen create windows-windows-10-latest-py3hosts file to add the relevant entry for salt (the master's IP address).salt-minion service.$ salt-key -a DESKTOP-MG2P5NV$ salt -Cv 'DESKTOP-MG2P5NV' state.sls packages.chocolatey test=True$ KITCHEN_LOCAL_YAML=kitchen.vagrant.yml bin/kitchen destroy windows-windows-10-latest-py3$ salt-key -d DESKTOP-MG2P5NV@dafyddj If you've got any additional ideas, they'd be welcome.
myii
commented
2 years ago I've added another comment in Slack, if anyone is following it there:
myii
commented
2 years ago @fgionghi @mvivaldi I've also reproduced the error on our Windows 8.1 box. I even tried different versions of Salt but didn't make any progress. Perhaps relative includes have never worked on Windows, in a master-minion set up.
It's time for this to be sent through as a bug report to the Salt repo itself. Most of the content you both have supplied here is useful to provide in the report there. I'll be happy to join in the discussion there as well -- please link it back to this issue.
In terms of this formula, the team will have to discuss how to proceed. @saltstack-formulas/wg Since relative includes aren't working on Windows, should we revert to using tplroot? Or is there some other method that we should consider. @ypid-geberit Your feedback will also be useful, since this is going to affect the changes you provided in #74. Could we find a new method, that uses only tpldir instead of tplroot?
fgionghi
commented
2 years ago Here the issue we have opened on the Salt repo: https://github.com/saltstack/salt/issues/61040 . Thank you.
myii
commented
2 years ago Thanks, @fgionghi -- I will add additional comments there.
Your setup
Formula commit hash / release tag
Release tag: 0.14.0
Versions reports (master & minion)
Salt master:
Salt minion:
Pillar / config used
Bug details
Describe the bug
When I run the command
salt skyros.pc.itc.it state.test packages.chocolatey, I get the error:and in the cache folder on the target (
C:\salt\var\cache\salt\minion\files\base\packages) the only file ischocolately.slswithout anymap.jinjaWe tried with different version of salt-minion, with different windows version (win10, win server 2016/2019) and with two different salt-master (same version of salt-master).
When I run
salt skyros.pc.itc.it state.test packagesin the cache folder on the target machine I have more files (for example themap.jinja) but not all files (thedefault.yamlis missing). In fact I get multiple errors regarding thedefault.yamlfile, like this:Expected behaviour
It should install the package
prometheus-windows-exporterusing chocolatey.Attempts to fix the bug
I submitted a PR that seems to fix the problem: #77