saltstack / salt

Software to automate the management and configuration of any infrastructure or application at scale. Get access to the Salt software package repository here:
https://repo.saltproject.io/
Apache License 2.0
14.09k stars 5.47k forks source link

Imported jinja without context in .sls are not cached #66485

Open baby-gnu opened 4 months ago

baby-gnu commented 4 months ago

Description

When doing an import of a Jinja from a .sls without passing context should be cached according to Jinja template but it seems to not be the case.

Setup

Install latest onedir on Debian 12 according to documentation

Please be as specific as possible and give set-up details.

Steps to Reproduce the behavior

Expected behavior

The rendering of /srv/salt/test-cache/slow-compute.jinja show be quick when loaded from /srv/salt/test-cache/test2.sls instead of during more than 10 seconds.

Screenshots

Versions Report

salt --versions-report (Provided by running salt --versions-report. Please also mention any differences in master/minion versions.) ```yaml Salt Version: Salt: 3007.0 Python Version: Python: 3.10.13 (main, Feb 19 2024, 03:31:20) [GCC 11.2.0] Dependency Versions: cffi: 1.16.0 cherrypy: unknown dateutil: 2.8.2 docker-py: Not Installed gitdb: Not Installed gitpython: Not Installed Jinja2: 3.1.3 libgit2: 1.7.2 looseversion: 1.3.0 M2Crypto: Not Installed Mako: Not Installed msgpack: 1.0.7 msgpack-pure: Not Installed mysql-python: Not Installed packaging: 23.1 pycparser: 2.21 pycrypto: Not Installed pycryptodome: 3.19.1 pygit2: 1.14.1 python-gnupg: 0.5.2 PyYAML: 6.0.1 PyZMQ: 25.1.2 relenv: 0.15.1 smmap: Not Installed timelib: 0.3.0 Tornado: 6.3.3 ZMQ: 4.3.4 Salt Package Information: Package Type: onedir System Versions: dist: debian 12.5 bookworm locale: utf-8 machine: x86_64 release: 6.1.0-12-cloud-amd64 system: Linux version: Debian GNU/Linux 12.5 bookworm ```

Additional context

Following #59441 for 3007.0 with simpler instruction by using salt["test.sleep"](10) instead of complicated ISO hash computation.

baby-gnu commented 1 month ago

Hello.

I made a test with a python script to see how jinja behave and when without context is used, the imported jinja file is not computed.

test.py script ``` python #!/usr/bin/python3 -u import os import time from jinja2 import Environment from jinja2 import FileSystemLoader CURRENT_DIR = os.path.dirname(__file__) loader = FileSystemLoader(CURRENT_DIR) env = Environment(loader=loader) env.globals["sleep"] = time.sleep templates = [ "test-import-with-context.jinja", "test-import-without-context.jinja", ] for template in templates: tmpl = env.get_template(template) for i in range(1, 4): print(f"Rendering {i} of {template}…", end=" ") t_start = time.time() output = tmpl.render() t_end = time.time() print(f"in {t_end - t_start} seconds\n{output}", "\n") ```
test-import-with-context.jinja ``` jinja {%- from "slow-compute.jinja" import slow_value with context -%} test-import-with-context : {{ slow_value }} ```
test-import-without-context.jinja ``` jinja {%- from "slow-compute.jinja" import slow_value without context -%} test-import-without-context : {{ slow_value }} ```
slow-compute.jinja ``` jinja {%- set time_sleep = sleep(10) %} {%- set slow_value = "value from slow-compute" %} ```

The with context import take 10 seconds to be rendered, and without is less than a milliseconds.

./test.py output ``` Rendering 1 of test-import-with-context.jinja… in 10.000969886779785 seconds test-import-with-context : value from slow-compute Rendering 2 of test-import-with-context.jinja… in 10.000410556793213 seconds test-import-with-context : value from slow-compute Rendering 3 of test-import-with-context.jinja… in 10.000433683395386 seconds test-import-with-context : value from slow-compute Rendering 1 of test-import-without-context.jinja… in 10.000469446182251 seconds test-import-without-context-01 : value from slow-compute Rendering 2 of test-import-without-context.jinja… in 0.00011181831359863281 seconds test-import-without-context-01 : value from slow-compute Rendering 3 of test-import-without-context.jinja… in 4.9114227294921875e-05 seconds test-import-without-context-01 : value from slow-compute ```