Support YAML args should all be able to be Jinja Templated

[gilesknap]

Ibek Schema Proposal

The problem

At present the Support Yaml Args must be set to type str if they are ever allowed to have Jinja templating.

This is counter productive because:

• type checking for non-string types is lost
• the decision on whether a particular arg may need templating is made at support module definition time instead of at ioc instance declaration time.

The solution

@GDYendell and @gilesknap propose that we make all arguments templatable as follows:

• string Args remain as they where and can contain arbitrary, multiple Jinja macros

• all other types will have a new scheme, below I use the example of int but this applies to float, bool, etc.

  • the type is declared as int in the Support YAML as before
  • but this generates a schema entry with Union type int | str
  • the str side of the Union will have a regex constraint that it must match
  • https://regex101.com/r/L3GUvk/1
  • regex: \"{{ .* ?\| ?int ?}}\"
  • example: "{{ myvalue * yourvalue + 1 | int}}"
  • additionally the ec ioc validate command will evaluate the Args and this means that you will get checks that the expression evaluates to something that can be cast to int.

Additionally

As part of this change we also propose that the Jinja rendering be run repeatedly as follows

• load all Args into the context
• replace every Arg with a jinja rendering of itself
• if there where any changes in this pass then perform another pass

This means that we can have nested references to rendered values. For example:

  - type: motorSim.simMotorAxis
    controller: controllerOne
    M: CS_M{{ADDR}}
    ADDR: "{{ 1 + axis_start | int }}"
    DESC: "CS Motor {{ADDR}}"
    home: 100
    CS_NUM: 3
    is_cs: true

Without a second pass of this we would have DESC = CS Motor {{ 1 + axis_start | int }}

The downside of this approach is that there is no possibility of using escaping to put {{ and }} in the final result.

@coretl I would like to get your sign off on this Additionally feature - what do you think?

[coretl]

I'm still slightly nervous of not being able to include escaping, seems like it might cause difficult to change issues down the line.

Also, how would a pathological case like this render?

- type: motorSim.simMotorAxis
  DESC: "{{ ADDR }}"
  ADDR: "{{ DESC }}"

[coretl]

I've got a potential idea, instead of populating the context with the values, populate them with deferred renderers, e.g.

@dataclass
Class DeferredRender:
    value: str
    context: Dict[str, Any]
    @lru_cache  # maybe?
    def __str__(self) -> str:
        return render(self.value, self.context)

context = {}
for arg _name, arg_value in args.items():
    context[arg_name] = DeferredRender(arg_value, context)

Then let Python's recursion size limit catch the recursive case

[GDYendell]

DeferredRender seems really neat.

[gilesknap]

that is pretty cool 🤓

[gilesknap]

Note that we don't want to do this for objects because we want to use dot-member.

[gilesknap]

The rendering as of 3.0.0b6 is all done during the model validator for the IOC instance. All typed fields are also allowed to be str as long as the str contains {{.*}}.

https://github.com/epics-containers/ibek/blob/list-arg2dict-param/src/ibek/ioc.py#L97-L122