[BUG] Single quotation mark in contents_pillar not escaped when using {% raw %}

[DaAwesomeP]

Description Single quotation marks in a {% raw %} block inserted via contents_pillar are not escaped properly. See example below.

Setup Pillar:

one:
  instances:
    - name: three
      well_known_acme_challenge: True
      files:
        fileone.json: |
          {
              "MY_VALUE": "value"
          }
        # The following YAML pillar contents has Jinja that should be ignored by Salt,
        # so I am using raw. This is a config file for another application.
        filetwo.yaml: |
          {% raw %}
          # ---
          # This file is managed by Saltstack on this machine. Manual modifications
          # to this file may be overridden when states are next applied.
          # ---
          somevalue:
            #   This comment's problem is it creates an error
            - anothervalue: >-
                {{ somejinjaexpression and
                  anotherjinjaexpression }}
           {% endraw %}

Salt:

{% set instance_index0 = loop.index0 %}
{% if (salt['pillar.get']('one:instances:' + (instance_index0 | string) + ':files', default={}) | length) > 0 %}
one_instances_{{instance.name}}_config:
  file.managed:
    - makedirs: True
    - names:
      {%- for file in salt['pillar.get']('one:instances:' + (instance_index0 | string) + ':files', default={}) %}
      - /path/to/{{instance.name}}/{{ file }}:
        # The following contents_pillar cannot be surrounded by raw because it relies
        # on templates for the value
        - contents_pillar: one:instances:{{ instance_index0 }}:files:{{ file }}
        - user: root
        - group: root
        - mode: '0640'
      {%- endfor %}
{%- endif %}

Error:

ERROR: Minions returned with non-zero exit code
myhost.example.com:
    Data failed to compile:
----------
    Rendering SLS 'base:app.one' failed: while parsing a flow mapping
  in "<unicode string>", line 28, column 160:
     ... acme_challenge': True, 'files': {'fileone.json': '{\n    "MY_VAL ... 
                                         ^
expected ',' or '}', but got '<scalar>'
  in "<unicode string>", line 28, column 2651:
     ... \n  #   This comment\'s problem is it creates an error ... 
                               ^

Please be as specific as possible and give set-up details.

• [x] on-prem machine
• [x] VM on KVM (TrueNAS SCALE)
• [ ] VM running on a cloud service, please be explicit and add details
• [ ] container (Kubernetes, Docker, containerd, etc. please specify)
• [ ] or a combination, please be explicit
• [ ] jails if it is FreeBSD
• [ ] classic packaging
• [x] onedir packaging
• [ ] used bootstrap to install

Steps to Reproduce the behavior See above configs.

Expected behavior Should insert file without error.

Screenshots N/A, see console output above.

Versions Report

salt --versions-report

```yaml Salt Version: Salt: 3006.6 Python Version: Python: 3.10.13 (main, Nov 15 2023, 04:34:27) [GCC 11.2.0] Dependency Versions: cffi: 1.14.6 cherrypy: unknown dateutil: 2.8.1 docker-py: Not Installed gitdb: Not Installed gitpython: Not Installed Jinja2: 3.1.3 libgit2: 1.7.2 looseversion: 1.0.2 M2Crypto: Not Installed Mako: Not Installed msgpack: 1.0.2 msgpack-pure: Not Installed mysql-python: Not Installed packaging: 22.0 pycparser: 2.21 pycrypto: Not Installed pycryptodome: 3.19.1 pygit2: 1.14.1 python-gnupg: 0.4.8 PyYAML: 6.0.1 PyZMQ: 23.2.0 relenv: 0.14.2 smmap: Not Installed timelib: 0.2.4 Tornado: 4.5.3 ZMQ: 4.3.4 System Versions: dist: debian 12 bookworm locale: utf-8 machine: x86_64 release: 6.1.0-18-amd64 system: Linux version: Debian GNU/Linux 12 bookworm ```

Additional context Possibly an additional option like contents_pillar_raw would help?

[DaAwesomeP]

Loosely related to #33319

[OrangeDog]

The error is due to invalid YAML. It has nothing to do with Jinja, thus {% raw %} will do nothing re. escaping.

Instead of trying to put formatted YAML strings into your pillar, you should leave it as structural data and use file.serialize, or render it with a template.

[DaAwesomeP]

My difficultly is that the final file written to disk should have the raw Jinja templates preserved, not evaluated (this is a config file for an application that uses Jinja in YAML as well).

My understanding was that data in a | in YAML is stored as a string, not YAML/dict. The error message shown indicated to me that it is being saved as a string but for some reason it is not escaping the contents properly?

I like the idea of simply making a template with {{ salt['pillar.get'] }} as the contents. In that case, how do I stop Jinja from evaluating in the expressions in string in the pillar?

[OrangeDog]

simply making a template with {{ salt['pillar.get'] }} as the contents

That is the same as using contents_pillar.

How do I stop Jinja from evaluating in the expressions in string in the pillar?

With {% raw %}, or changing the rendering pipeline for the whole file (e.g. #!yaml at the top).

Your pillar is working fine, though is overly complicated. The problem is (I think) that you are iterating the values, not the keys. So your Jinja-rendered state has random chunks of YAML pasted into it.