Is it possible to keep double-braces in a template?

[billdenney]

Thanks for writing your package

I'm wanting to make a 2-stage template (perhaps referred to as a meta-template). In the first time through the template, I'd like to replace some of the values and leave others. In the second time through, I'd like to replace the rest.

While writing this issue into a reproducible example, I found the way to do it is by assigning NULL as follows:

For values:

starttmpl <- tmpl("{{ replaced_during_the_first_pass }} {{ replaced_during_the_second_pass }}")
intermediatetmpl <- tmplUpdate(starttmpl,
                               replaced_during_the_first_pass="{{ prepared_during_the_first_pass }}",
                               replaced_during_the_second_pass=NULL)
# "{{ prepared_during_the_first_pass }} {{ replaced_during_the_second_pass }}"
final <- tmplUpdate(intermediatetmpl,
                    prepared_during_the_first_pass=1,
                    replaced_during_the_second_pass=2)
# "1 2"

And for functions:

starttmpl <- tmpl("{{ replaced_during_the_first_pass }} {{ replaced_during_the_second_pass(x) }}")
intermediatetmpl <- tmplUpdate(starttmpl,
                               replaced_during_the_first_pass="{{ prepared_during_the_first_pass }}",
                               replaced_during_the_second_pass=function(x) NULL)
# "{{ prepared_during_the_first_pass }} {{ replaced_during_the_second_pass(x) }}"
final <- tmplUpdate(intermediatetmpl,
                    prepared_during_the_first_pass=1,
                    x=2,
                    replaced_during_the_second_pass=function(x) x^2)
# "1 4"

Can you please update the documentation to tmplUpdate to indicate something like:

In Details:

"Assigning a value of NULL to a name or a function that returns NULL to a function to be evaluated will leave the value within the template. See examples for how to use this feature."

In Examples for tmplUpdate:

# Keep a named value in the output
tmpl("{{ keep }}", keep=NULL) # "{{ keep }}"
# Keep a function in the output
tmpl("{{ keep(x) }}", keep=function(x) NULL) # "{{ keep(x) }}"

[billdenney]

I spoke too soon. That method doesn't appear to work reliably, and there is a bug if a value is assigned to NULL where that value is ignored and other values are moved up the list of named values:

library(templates)
tmpl("{{ first }} {{ second }}", first = NULL, second = 2)
#> 2 {{ second }}
tmpl("{{ first }} {{ second }} {{ third }}", first = NULL, second = 2, third = 3)
#> 2 3 {{ third }}

[wahani]

Hi, I am happy that the package is of some use.

I was not of aware of the behavior you describe. I need to fix that. It appears that adding NULLs leads to unexpected behavior. Thanks for the taking the time to figure that out.

When I wrote the package I decided to not allow -- at least I thought I wouldn't -- for partial evaluation of a template. This was too much trouble because it is very hard to find out if the user does not want to evaluate a particular chunk, or if arguments are missing or not spelled correctly. I haven't thought of using a NULL as an indicator. To delay the evaluation of a template I always envisioned to use functions. Maybe this code helps:

metaTemplate <- function(t, ...) {
  t <- tmpl(t)
  args <- list(...)
  function(...) {
    do.call(tmpl, c(t, args, list(...)))
  }
}

template <- metaTemplate(
  # tmpl("{{ first }} {{ second }} {{ third }}"), second = 2, third = 3
  "{{ first }} {{ second }} {{ third }}", second = 2, third = 3
)

template(first = 1)

Something else you can try with slightly different semantics:

tmplDelayed <- function(t, ...) {
  t <- tmpl(t)
  args <- list(...)
  function(..., eval = FALSE) {
    if (eval) do.call(tmpl, c(t, args, list(...)))
    else do.call(tmplDelayed, c(t, args, list(...)))
  }
}

template <- tmplDelayed("{{ first }} {{ second }} {{ third }}")
template <- template(second = 2)
template <- template(first = 1)
template <- template(third = 3)
template(eval = TRUE)

This second approach is somewhat more generic since you are not restricted to only two stages in which you accumulate the arguments. Please let me know if any of these solutions are helpful. We can think about adding something like the above to the package. However I am not sure that using NULLs to indicate to not evaluate particular chunks is the way I would want to proceed. If you have the time, what is your first thought, or what do you expect when you pass in a NULL value? I was thinking an empty string might be useful. But maybe I am wrong.

[wahani]

Sorry, I made a mistake in my example above. Should be

template <- metaTemplate(
  "{{ first }} {{ second }} {{ third }}", second = 2, third = 3
)

template(first = 1)

[billdenney]

Thanks for the detailed response and the thoughtful suggestions.

What I'm actually trying to do is to inject additional template parts with the first pass and then fill in those injected parts. As a more direct example of what I'm trying to do:

meta_template <- tmpl("{{ first }} {{ second }}")
template <- tmplUpdate(meta_template, second="{{ a }} {{ b }}")
final <- tmplUpdate(template, a=1, b=2, first=3)

Perhaps a way to make the package work with multiple passes is to have an argument like ".ignore.errors". ".ignore.errors" would be set to FALSE by default, but if set to TRUE, then if the execution of a code block failed it would be left as-is in the final template.

So, the call would look something like:

meta_template <- tmpl("{{ first }} {{ second }}")
template <- tmplUpdate(meta_template, second="{{ a }} {{ b }}", .ignore.errors=TRUE)
final <- tmplUpdate(template, a=1, b=2, first=3)

Two other alternatives are:

If the user has a specific number of passes in advance, they could escape the braces with something like:

"{{{{ first }}}} {{ second }}"

Or, you could enable it to be generically left as is by using a tmplAsis function which could be defined like:

tmplAsis <- function(...) {
  match.call()
}

then they could do:

meta_template <- tmpl("{{ first }} {{ second }}")
template <- tmplUpdate(meta_template, second="{{ a }} {{ b }}", first=tmplAsis)
final <- tmplUpdate(template, a=1, b=2, first=3)

(Though I don't think match.call by itself will provide the output as needed with the current package.)

[wahani]

This is probably not really what you are looking for but it is a certain way to think about the problem:

library(templates)

tmplSprintf <- function(t, ...) {
  templates:::tmplConstructor(
    sprintf(t, ...),
    attr(t, "envir")
  )
}

metaTemplate <- tmpl("{{ first }} %s")
template <- tmplSprintf(metaTemplate, "{{ a }} {{ b }}")
tmplUpdate(template, a = 1, b = 2, first = 3)

Just elaborating on the issue: What we have to keep in mind is, that a template is represented as a character. When the evaluation takes place I use regular expressions to identify chunks. These chunks are then evaluated in the context of the arguments passed to tmplUpdate. Although it is possible to implement something like partial updates of a template it is not that easy. It means to further parse each chunk and figuring out if it is left alone or evaluated. Since these chunks are then regular R expressions Rs lexical scoping rules are applied. So just by not specifying an argument it is hard to see if this means that a chunk should not be evaluated since we may just refer to an object elsewhere. Hence we need a data type indicating a partial update. This brings us to something similar to your suggestion with tmplAsis. But since we can have chunks like {{ first(second) }} it would mean that any of first or second can indicate a partial update.

I need to think a bit on how and if this is going to work. If you have an idea and time to work on a solution feel free to make a PR.

[billdenney]

What I ended up doing was similar to your suggestion. I did:

metaTemplate <- tmpl("{{ first }} [[ second ]]")
template <- gsub("[[ ", "{{ ", gsub(" ]]", " }}", tmplUpdate(metaTemplate, first=1), fixed=TRUE), fixed=TRUE)
tmplUpdate(template, second=2)

I think that my simplest preference is to allow errors to pass through unmodified if .errors="ignore". That seems to be both simple, and will maintain user's intent. In your example, {{ first(second }} with tmplUpdate("{{ first(second) }} {{ third }}", first=1, third=3, .errors="ignore") would yield "{{ first(second) }} 3" which seems logical and would allow multi-pass.

For more granular control, it could have .errors="error", .errors="warning", and .errors="ignore" where the error could be converted to a warning so that there is also user notification along the way.

I'll make this into a pull request.

[wahani]

Okay, great.

I would prefer a solution where we have an additional function (like tmplPartialUpdate) which implements this feature, although some additional arguments are also okay. I just reviewed the code and I think we need to add a new evaluator. The current version:

evaluator <- function(x, envir, enclos) {
  # x (chunk without {{) so '{{ first }}' -> first
  # envir (list | environment) the arguments we pass down, so 'first = 1'
  # enclos (environment) the environment in which the template has been created
  flatmap(x, function(sexpr) {
    as.character(eval(parse(text = sexpr), envir = envir, enclos = enclos))
  })
}

takes each chunk and evaluates it inside the environment defined by the arguments we pass in. Hence the additional evaluator needs to be able to return the elements in x wrapped in {{. So when we define envir = list(second = 2) it should return c("{{ first }}", "2"). The input would be c("first", "second").

[wahani]

@billdenney I uploaded a version to fix the issue when passing NULL values to tmplUpdate: 3c3cea014972b8d8a97af788e3a570665c3c75c6. Can you verify that this is now working after installing the version from GitHub?

[billdenney]

That fixes the NULL issue for me. Thanks!

[wahani]

If this becomes relevant again or anybody want's to make a PR: please reopen.