Use Jinja/Curly brackets inside YAML configs

[kingo55]

The modular builder could be more expressive if we used curly brackets to inject JS/CSS etc than if we hard-coded this logic into specific fields:

https://jinja.palletsprojects.com/en/2.11.x/templates/

It works quite well in Jinja, dbt, Home Assistant and other projects, so it might work well here too. Rather than:

state: staging
sampleRate: 0
id: w474
name: Some test name
css: shared.css
recipes:
  '0':
    name: Control
  '1':
    name: Treatment1
    js: 1.js
trigger: trigger.js

We could have something like:

state: staging
sampleRate: 0
id: w474
name: Some test name
css: {{ refCss('some-file.css') }}
recipes:
  '0':
    name: Control
  '1':
    name: Treatment1
    js: {{ ref('1.js') }}
trigger: {{ ref('trigger.js') }}

This would allow us to support new variable types (e.g. JSON) or fields as they come available e.g. custom functions passed into the option keys, like decisionAdapter, storageAdapter and whatnot.

This would change how we write all experiments' YAML files though, so perhaps we need to think this through properly and map out migration.

[allmywant]

@kingo55 "curly brackets to inject JS/CSS" looks very useful, the only problem is: currently we use CleanCSS to minify the css before injection, not sure if we can do it using Jinja templates.

[kingo55]

Could we parse the contents of the curly brackets into a JS function?

e.g. {{ injectCss('some.css') }}

And the macro reads/parses/minifies the CSS?

[allmywant]

@kingo55 It looks like we can write the template like this:

state: staging
sampleRate: 0
id: w474
name: Some test name
css: injectCss('{{ 'some-file.css' }}')
recipes:
  '0':
    name: Control
  '1':
    name: Treatment1
    js: ref('{{ '1.js' }}')
trigger: ref('{{ 'trigger.js' }}')

Then Jinja outputs: css: injectCss('some-file.css') js: ref('1.js')

Then we can call injectCss function in modular builder to read and minify the CSS files.

[kingo55]

That's good to know @allmywant - could users run any arbitrary code in here?

[allmywant]

@kingo55 It could be, we can limit the the functions in curly brackets must be the ones we supported and throw error if the function name is unknown.

[kingo55]

@allmywant - it could be a feature because there's other easier places to run arbitrary code...

This might just enable some unique features we're not aware of just yet. E.g. Fetching a dynamic sampleRate key during ModularBuild, so we don't have to pollute the git history with daily/hourly sampleRate key changes.

[allmywant]

@kingo55 Good idea! I think we can do it like that.