tanbro / pyyaml-include

yaml include other yaml
https://pypi.org/project/pyyaml-include/
GNU General Public License v3.0
77 stars 20 forks source link

Jinja2 support #12

Closed elvis2 closed 4 years ago

elvis2 commented 4 years ago

Hello,

How likely are you to support jinja templates?

I have YAML templates that have jina logic in them. For example:

 containers:
    - name: "{{application_name}}"
      image: "{{business_unit}}-docker-{{environment}}.artifactory.svc.aws.gartner.com/{{application_name}}:{{version}}"
      imagePullPolicy: IfNotPresent
      ports:
        - containerPort: 8080
          protocol: TCP
      resources:
        limits:
          memory: {{ vars.eks.memory_limits if vars.eks.memory_limits else '512mb' }}

The last line of the code, is a jinja2 expression.

The probably is, I run the template through the pyyaml-include, to get a full single document from multiple templates. Then, I run the single template through the template engine.

If this:

memory:" {{ vars.eks.memory_limits if vars.eks.memory_limits else '512mb' }}"

when pyyaml-include operates on this line, you get:

memory: '{{ vars.eks.memory_limits if vars.eks.memory_limits else ''512mb'' }}'

And the jinja2 engine can't understand it. What jinja2 is expecting is this:

memory: {{ vars.eks.memory_limits if vars.eks.memory_limits else '512mb' }}

Are you willing to support a jinja2 template language?

Thanks

tanbro commented 4 years ago

I have another project jinjyaml, it provides pyYAML parser and loader for Jinja2 template express/object

But i haven't tried to use them together.

elvis2 commented 4 years ago

Thanks, I'll keep an eye on your new project.

To make this more clear. I was using the pyyaml-include first, then the jinja2 render last. I found out the yaml dump from the pyyaml include module was changing the quotes to where the jinja2 enginer render couldn't process it.

Instead, I force the jinja2 render first, for all templates that will be used by pyyaml include, then run that include module last. It works as expected now.

Thanks

tanbro commented 4 years ago

The project is based on pyYAML Constructor, which is a part of it's YAML parser.

When including another YAML file, it (the file) will be parsed into YAML objects(Scalar, Sequence, or Mapping object), then put to the place where the "!include" tag is.

So, the Jinja2 template expressions won't be recognized during the including process.

for example, a file like:

{% for employee in department.employees %}
- name: employee.name
  age: employee.age
{% endfor %}

can not be included, cause it's legal for Jinja, but illegal for YAML. Unless we render it before including.

For a "render after include" usage, i think jinjyaml is helpful.

It provides Jinja2 template constructor/tag. We can rewrite Jinja2 template in above example to a YAML with a special tag:

!jinja2 |
{% for employee in department.employees %}
- name: employee.name
  age .employee.age
{% endfor %}

Such a fragment will be parsed to a jinjyaml.JinjyamlObject object (note that we shall add jinjyaml constructor to FullLoader first). Thus we can render it later.