[BUG] Compiled template temp files not being cleaned up

gpezza2 commented 1 year ago

Description When using file tree external pillar data that needs to be rendered through the Jinja/YAML renderer, Salt uses a temp file of the form __salt.tmp.xxxxxx from the mkstemp utility through the compile_template_str function. However, the temp file is then not cleaned up, leading to a many orphaned temp files in /tmp

Setup Use ext_pillar config option to define external pillar data using the file_tree method as outlined here: https://docs.saltproject.io/en/latest/ref/pillar/all/salt.pillar.file_tree.html#salt.pillar.file_tree.ext_pillar

 - file_tree:
      root_dir: /srv/host-pillar
      follow_dir_links: False
      keep_newline: False
      render_default: jinja|yaml
      renderer_blacklist:
        - gpg
      renderer_whitelist:
        - jinja
        - yaml
      template: True

the external pillar root is organized as such:

/srv/host-pillar/hosts/{hostname}/host-pillar

[X] on-prem machine
[ ] VM (Virtualbox, KVM, etc. please specify)
[ ] VM running on a cloud service, please be explicit and add details
[ ] container (Kubernetes, Docker, containerd, etc. please specify)
[ ] or a combination, please be explicit
[ ] jails if it is FreeBSD
[X] classic packaging
[ ] onedir packaging
[ ] used bootstrap to install

Steps to Reproduce the behavior Setup a file_tree external pillar path, have it use the jinja|yaml rendered default with template: true. Then check your /tmp, it should start filling up with __salt.tmp.xxxx files that are a copy of the data in your external pillar files everytime a highstate runs.

Expected behavior The expected behavior is that after the tmp file is used to compile the template into high data structure, the temp file should be cleaned up.

Versions Report

salt --versions-report

(Provided by running salt --versions-report. Please also mention any differences in master/minion versions.) ```yaml Salt Version: Salt: 3005.1 Dependency Versions: cffi: Not Installed cherrypy: unknown dateutil: Not Installed docker-py: Not Installed gitdb: Not Installed gitpython: Not Installed Jinja2: 2.11.1 libgit2: Not Installed M2Crypto: 0.35.2 Mako: Not Installed msgpack: 0.6.2 msgpack-pure: Not Installed mysql-python: Not Installed pycparser: Not Installed pycrypto: Not Installed pycryptodome: Not Installed pygit2: Not Installed Python: 3.6.8 (default, Nov 16 2020, 16:55:22) python-gnupg: Not Installed PyYAML: 3.13 PyZMQ: 18.0.1 smmap: Not Installed timelib: Not Installed Tornado: 4.5.3 ZMQ: 4.1.4 System Versions: dist: centos 7 Core locale: UTF-8 machine: x86_64 release: 3.10.0-1160.83.1.el7.x86_64 system: Linux version: CentOS Linux 7 Core ```

Additional context I plan to submit a PR shortly with a suggested solution.

welcome[bot] commented 1 year ago

Hi there! Welcome to the Salt Community! Thank you for making your first contribution. We have a lengthy process for issues and PRs. Someone from the Core Team will follow up as soon as possible. In the meantime, here’s some information that may help as you continue your Salt journey. Please be sure to review our Code of Conduct. Also, check out some of our community resources including:

There are lots of ways to get involved in our community. Every month, there are around a dozen opportunities to meet with other contributors and the Salt Core team and collaborate in real time. The best way to keep track is by subscribing to the Salt Community Events Calendar. If you have additional questions, email us at saltproject@vmware.com. We’re glad you’ve joined our community and look forward to doing awesome things with you!

whytewolf commented 1 year ago

Closed with #63932

saltstack / salt

[BUG] Compiled template temp files not being cleaned up #63724