Allow to force data types in YAML after jinja expansion

[yajo]

Proposal: Allow to force data types in YAML after jinja expansion

Author: Jairo Llopis <@Yajo>

Date: 2017-09-19

• Status: New
• Proposal type: (core design, process, ?)
• Targeted Release: <future release #, ex. 2.2>
• Associated PR: <link to GH PR in ansible/proposals if PR was submitted>
• Estimated time to implement: <X days, weeks, etc.>

Motivation

Sometimes you cannot use all of ansible+jinja power if the module did not define the right data type.

Problems

What problems exist that this proposal will solve?

• https://github.com/ansible/ansible/issues/17992 and all its related issues.

Solution proposal

As explained in https://github.com/ansible/ansible/issues/15249#issuecomment-330481415, Ansible should allow and parse YAML application-specific local tags that force a type cast after the value has been jinja2-evaluated

YAML itself has already type casters such as !!int or !!float (double !!), but those happen before jinja2, and thus they produce an error.

Basically one should be able to add !str, !int or !float (notice the single !) to any variable, either with or without jinja2 expansion, and it should get correctly type-casted after processing the jinja2 filters.

For instance, this should then be a valid workaround for https://github.com/ansible/ansible/issues/30548:

- docker_service:
    pull: true
    state: present
    project_name: test
    definition:
      version: "2.2"
      services:
        dind:
          image: docker:dind
          cpus: !float "{{ ansible_processor_cores / 2 }}"

Testing (optional)

A simple debug task would be enough to test it IMHO.

Documentation (optional)

Of course, this should be documented.

[bcoca]

This assumes templating happens before or during the YAML processing, it happens well after, so the YAML type would error out as templates are always strings, by definition.

Type management is then done either in Jinja2 via filters |int or |float and/or in the module option definitions themselves (we call this argspec) name=<option name>, type='integer', which should cast to the correct type if possible.

for your case it would look like this:

 cpus:  "{{ (ansible_processor_cores / 2)|float }}"

Sadly this can be 'lost' when templating again, so you always want to use it on 'consumption' and not on 'definition', we have a PR to jinja2 itself to be better about preserving type but that will take time.

[dagwieers]

The PR to preserve type in Jinja2 is https://github.com/pallets/jinja/pull/708

[yajo]

Well, the YAML tag (what @bcoca calls type) errors out when it is a built-in one (2 !!), but YAML allows applications to define local tags (1 !). Ansible can map those tags to classes/functions that perform lazy type transform (after jinja evaluation), so the type conversion happens in the correct time.

As an example, you can check out how Odoo implements this.

[sivel]

The problem as @bcoca mentioned with your specific implementation proposal, is that YAML and jinja parsing happen at different times. So trying to use YAML tags, doesn't work as expected.

What happens is that when YAML is parsed the !float tries to run on a value of "{{ ansible_processor_cores / 2 }}"

The above in your example tries to turn the literal string "{{ ansible_processor_cores / 2 }}" into a float at YAML parsing time which fails.

The PR that @dagwieers mentions is the appropriate solution.

[yajo]

Well, I said lazy type transform for some reason. But indeed, pallets/jinja#708 is the best bet.

[bcoca]

@Yajo, closing this proposal as per above. The real place to implement this is in Jinja2 itself.