Variables with dynamic, interpolated dereferenced values

Replicate

Consider the following Markdown (e.g., README.md):

![Logo]($application.directory.images$/logo.png)
$application.title$
===
$application.description$
![Screenshot]($application.directory.images$/screenshot.png)

Along with the following standalone YAML file (neither _config.yaml nor document front matter):

application:
  title: Application
  directory:
    images: images
  description: $application.title$ is almost as amazing as Liquid.

Prior to the markdown (e.g., README.md) being displayed, some recursive pre-parsing and substitution would help eliminate some duplication within the document.

Expected Results

![Logo](images/logo.png)
Application
===
Application is almost as amazing as Liquid.
![Screenshot](images/screenshot.png)

Actual Results

No substitution is preformed, so the README.md file is displayed as:

![Logo]($application.directory.images$/logo.png)
$application.title$
===
$application.description$
![Screenshot]($application.directory.images$/screenshot.png)

Additional Details

Using YAML as a bridge between documentation and applications is a slick way to avoid duplication. Since the YAML file is machine-readable, it would be trivial for an application to read its name from the same YAML source as the documentation. Changing the app name would be a simple matter of updating a single YAML file, which simultaneously updates all references in the documentation.

This could be an option passed into Liquid to maintain backwards compatibility.

Using _config.yaml is insufficient here because of namespace clashes. There are a couple of options:

Liquid can read all the Markdown files alongside _config.yaml and pass them through a pre-processor; or
_config.yaml can specify the pre-processor file(s) to use against all, or specific, Markdown files.

The first is easier, the second is more flexible.

Regarding Collections

Consider:

collections:
  my_collection:
    foo: bar

This would mean changing $application.title$ to $collections.my_collection.application.title$ ; such a prefix wouldn't necessarily make sense in other contexts (external to Jekyll/Liquid), and would require a lot of effort to refactor variable names across many Markdown documents (and verify that the refactor was performed correctly).

Also, part of the problem is that the variables must be able to reference other variables, with a variable interpolator to extract the correct dereferenced value (see $application.description$ , above). (Aside, this aspect often gets overlooked whenever I describe the problem.)

The collection attributes almost fits the bill, but has the stringent requirement of not being a standalone YAML file (i.e., must be part of Markdown front matter).

Variable Name Delimiters

Another issue is that it should be possible to define the variable delimiters and separators, such as:

{{ application.title }} -- {{ and }} with a . separator;
$application.title$ -- $ and $ with a . separator; or
optionally, `r#x( v$application$title )` -- `r#x( v$ and )` with a $ separator, which would be useful for evaluating inline R statements, evaluated using knitr or RinRuby.

Not a Template Language

What is being proposed is not another template language, but two related ideas.

First, variable substitution for arbitrary definitions using a dot-separated notation and recursive interpolation. Second, the ability to set a prefix and suffix delimiter used for finding and replacing variables with a resolved value map.

I've developed both of these in Java. The first is about 19 lines of code. The second requires a kind of variable decorator pattern, which is also fairly trivial in terms of LoC (fewer than 50 lines).

Code that deals with variable substitution sometimes hard-codes variable delimiters in multiple places, which is a violation of DRY. Using a constants is better practice, but still violates DRY when those constants are referenced in multiple places, rather than being constructed dynamically using a factory pattern based on some arbitrary criteria (such as a file name extension).

I can't quite figure out what you're getting at and how it's related to Liquid. As far as I can tell you are suggesting three things:

YAML definition of Liquid variables
YAML parsing with interpolated variables from the same file
Alternate syntax for Liquid interpolation ($ instead of {{)

The first one is out of scope. If you need this, use an existing YAML parser and pass the resulting hash into Liquid.

The second idea would be cool but I believe is also out of scope. Implementing this would require making Liquid aware of YAML syntax. If you feel that such a feature should exist, try raising it with a YAML project.

These first two ideas seem like they would be appropriate for an application framework rather than the templating language itself.

The third idea would inevitably conflict with existing templates. We could consider making it configurable, but it makes the parser code more complicated.

The second idea would be cool but I believe is also out of scope. Implementing this would require making Liquid aware of YAML syntax. If you feel that such a feature should exist, try raising it with a YAML project.

Could also be a generic "data source" with a factory that creates the appropriate reader based on a filename extension. For example, settings_schema.json would invoke the JSON reader while settings_schema.yaml would instantiate the YAML reader. Same for TOML, XML, and so forth.

The third idea would inevitably conflict with existing templates. We could consider making it configurable, but it makes the parser code more complicated.

If there's only one spot in the parser where {{ and }} are referenced, then the code wouldn't be that much more complicated. A factory or kind of inverse decorator pattern would be trivial. If, however, there are multiple locations where the delimiters are referenced, that's probably a sign that the parser could use some refactoring, depending on whether it's a hand-built parser or generated using a grammar. I'm guessing the former, because if it was the latter, it'd be trivial. ;-)

It would have to be a configuration option, though, as you noted.

I can't quite figure out what you're getting at and how it's related to Liquid.

Some background. I sent a note to GitHub staff asking if it was possible to use interpolated variables from an external data source within Markdown. They said that it should be possible because they are using Jekyll and Liquid. I posted a similar feature request to the Jekyll team and they suggested that this feature request was more suited to Liquid.

Now the suggestion is to pass the request along to a YAML team, when YAML has nothing to do with the general idea. The variables could be defined in any file format; it's trivial to machine convert from one structure to another.

Big Picture

Looking to write technical documentation that references variables defined in external files and have the dereferenced values interpolated/woven into an output document format. When the output document is rendered as HTML for a GitHub project page, for example, the variable references should be substituted with values from those definition files.

Example

For example, I'd like to write the following inside README.md:

# $application.name$ template engine

## Introduction

$application.name$ is a template engine which was written with very specific requirements:

* It needs to be non evaling and secure. $application.name$ templates are made...

## Why you should use $application.name$

I might have another file called CONTRIBUTING.md that includes:

# Workflow

* Fork the $application.name$ repository

Plus one more file named variables.json, variables.yaml, or variables.xml -- the format is not relevant, but for the purposes of an example, I'll use YAML for its simplicity:

application:
  name: Liquid

Both Markdown files would generate the expected output with "Liquid" being substituted wherever $application.name$ appears. This would require some interpolation because the variables should be able to reference other variables. For example, the variables.yaml file could also be:

application:
  name: Liquid
  template: $application.name$ Template

Any input document that references $application.template$ would have Liquid Template woven within the rendered output document.

I am not familiar with how Jekyll works but can you not already interpolate into markdown files? And does it not already provide a way to define variables from file?

Liquid knows nothing about configuration files or anything of the like - it's a library that provides an API for interpolating the parameters to the render method into the template provided to the parse method. This repo contains the language implementation and not much more.

Looks like it's not possible with the current architecture.

It's really not that difficult to leverage the chain-of-command design pattern to give multiple processors the chance to parse a document before generating a final output document. A simple Aho-Corasick algorithm as one of the links in the processing chain could provide a mechanism to search and replace variables. This would then pass the document along to the next processor in the chain.

Shopify / liquid