Expose Universal Filters and Shortcodes in configuration API (to load into custom engines that support them)

[noelforte]

Is your feature request related to a problem? Please describe.

It would be nice to have access to a list of Universal Filters and Shortcodes in the configuration API. I just released eleventy-plugin-vento v1 to add support for the Vento templating language by @oscarotero. Vento has the ability to use custom filters and tags, but retrieving these things from Eleventy via a plugin feels a bit brittle.

Currently my implementation is approximately as follows (shortened for brevity):

// Create empty sets to hold user defined functions
const filters = {};
const helperFunctions = {};

for (const fn in eleventyConfig.javascriptFunctions) {
  if (
    eleventyConfig.liquidFilters[fn] &&
    (eleventyConfig.nunjucksFilters[fn] || eleventyConfig.nunjucksAsyncFilters[fn])
  ) {
    filters[fn] = javascriptFunctions[fn];
  } else {
    helperFunctions[fn] = javascriptFunctions[fn];
  }
}

and then farther down...

eleventyConfig.addExtension('vto', {
  async compile(inputContent, inputPath) {
    return async (data) => {
      // Rebind functions to page data so this.page,
      // this.eleventy, etc works as intended
      for (const helper in helperFunctions) {
        helperFunctions[helper] = helperFunctions[helper].bind(data);
      }

      // Rebind filters the same way
      for (const filter in filters) {
        VentoLib.filters[filter] = filters[filter].bind(data);
      }

      // Merge helpers into page data
      data = { ...data, ...helperFunctions };
    };
  },
});

Note that this does not include shortcodes, as the helperFunctions object seems to do the trick to handle single shortcodes.

Describe the solution you'd like

It would be great to be able to retrieve all filters at once via an eleventyConfig.universalFilters call to make loading in easily.

As for single and paired shortcodes, it would be great to have similiar properties (eleventyConfig.universalShortcodes/eleventyConfig.universalPairedShortcodes) to achieve a similar result.

It would also be nice to have some sort of mechanism for adding page context data to filters/shortcodes in custom templating languages, although I'm not quite sure what that would look like beyond the .bind(data) technique I used above since the bundled engines Eleventy ships with seem to rely on some internal methods to manage adding context to filters and shortcodes. I'm also curious if Eleventy does any sort of caching work when exposing this.page and this.eleventy contexts when running shortcodes and filters for bundled engines and if plugins could take advantage of that pipeline in some way.

Describe alternatives you've considered

As far as I know, there aren't any documented mechanisms to retrieve Universal Shortcodes/Filters. While there's plenty of ways to get data into the data cascade for custom templating languages (getData(), getInstanceFromPath()), retrieving data from the data cascade for custom templating languages outside of Eleventy's internal methods looks to be a bit harder to do.

My current approach gets close enough; merging javascriptFunctions() into the data cascade and binding page data to this since Vento can run JS directly. However, this doesn't handle paired shortcodes/custom tags, which is a feature I'd like to eventually have in Vento templates within Eleventy. This would require more filtering of eleventyConfig.javascriptFunctions in order to extract the shortcodes and manually set up the integrations with Eleventy and Vento, which might be out-of-scope for what a plugin can do.

Additional context

With the deprecation of HAML, Handlebars, EJS, and Pug in core for v3 and some of them moving to plugin-land, I realize that I might be ahead of the curve here.

I'd be curious to know if there's a preferred way to integrate a filter-and-custom-tag-capable language with Eleventy's core functionality inside of a plugin. If not, will there be plans to do so to accommodate some of the templating languages being moved into plugins for v3?

[noelforte]

Adding a reference to discussion for Eleventy's official pug plugin (#3081), curious to see what @Zearin gets up to if/when they bring support for pug filters (or tags, if Pug has that sort of thing) to their plugin.

[monochromer]

+1. I create plugin for edge.js template engine and need somehow differentiate shortcodes, filters and functions, etc.

[noelforte]

I reviewed #3081 again, specifically this comment, that mentions:

I asked zachleat on Mastodon about using Eleventy v2’s class Pug extends TemplateEngine as the basis for a plugin in Eleventy v3, but he told me that wasn’t possible because it’s incompatible with the Plugin API.

I don’t know Eleventy’s internals enough to figure out how to recreate its operation piece-by-piece in a plugin.

...which is the same conclusion I came to in my original post. While it would be nice to have some deeper integration with Eleventy's internal TemplateEngine API for language extensions that overlap with Eleventy's features, it seems like I was correct with my assumption that these things may be out of scope for a plugin.

I know @zachleat is busy locking down features and trying to get 3.0 beta out so I don't want this to derail that. I am curious if there's a dedicated place to discuss providing support for other template languages in Eleventy, because I'm pretty sure that @monochromer, @Zearin and I's needs are very similar here. Thanks!

[zachleat]

Milestoned this one to 3.0, I think it’s important to get something better here.

[noelforte]

@zachleat Wow, that's awesome to hear! Let me know if I can help out in any way, testing, implementation, etc. I'm sure Vento, edge.js, Pug, etc. aren't the last we'll have for novel template engines so having this will be huge.

[zachleat]

The following configuration API additions are shipping in 3.0.0-alpha.15 and apply to Universal Filters and Universal Shortcodes only:

eleventyConfig.getFilter(name); // already existed
eleventyConfig.getFilters();

eleventyConfig.getShortcode(name);
eleventyConfig.getShortcodes();

eleventyConfig.getPairedShortcode(name);
eleventyConfig.getPairedShortcodes();

Simultaneously, this is mostly academic as these all feed back into JavaScript Functions but we’re also making a baby step to start separating filters, shortcodes, and paired shortcodes for 11ty.js JavaScript templates (while maintaining existing backwards compatibility).

eleventyConfig.addJavaScriptFunction(); // already exists, populated by both shortcodes and filters
eleventyConfig.addJavaScriptFilter(); // via universal filters and addFilter
eleventyConfig.addJavaScriptShortcode(); // via universal shortcodes and addShortcode
eleventyConfig.addPairedJavaScriptShortcode(); // via universal paired shortcodes and addPairedShortcode

[noelforte]

Zach, you are awesome, thank you so much for this. Once alpha.15 ships I'll take a shot at integrating these API additions and will flag any bugs that come up in the process.

[pauleveritt]

Hi, I'm working on TSX as a language. I'd be interested in making a plugin. As the linked ticket shows, I ran into similar points made here.

One extra point: bundles feel like a new point not mentioned above. For example, a shortcode wants to get some CSS injected into <head>. There's some kind of deferred evaluation going on that seems beyond normal shortcode/filter evaluation.

As noted above, many of the filters and shortcodes are just plain functions whose data only comes from stuff passed in. But some implicitly get config from the closure. Not sure if other bundled shortcodes/filters get other state from the closure or other places? The examples imply the only inputs are: passed in values and eleventyConfig.

Last point: I'd love if all the bundled shortcodes and filters were importable. Currently they are arrow functions defined inside a config setup -- even if the function doesn't use config.

Instead, I'd prefer standalone functions that can take an optional config argument. The bundled ones can have an arrow function that registers them and passes in the config.

But TSX could just import the function -- autocomplete, generate import, navigation, intelligence, etc. The TSX component that calls the shortcode would have to pass in the config, for the places where a shortcode actually needed it.

[noelforte]

@zachleat I pulled the latest HEAD directly from GitHub to test and it's working great so far. One question though: is there a preferred way to handle a shortcode that uses this in the function body? (ie this.page)

For example: if I try to run a filter/shortcode containing this.page.url, I get an error because this.page is undefined. I could reuse the .bind() approach I've had (or .call()) but wanted to see if there was a more official reccomendation for adding context to shortcodes, filters, etc.

I tried to reference how the internal Liquid Engine handles context but it looks like it relies on internal methods. Any suggestions for handling this from plugin-land?

[pauleveritt]

@noelforte I'd be interested to see if you can get the eleventy-plugin-bundle CSS stuff working, where one of your "subcomponents" (filter, shortcode) puts stuff in the bundle.

I have a theory that this doesn't work for JavaScript Templates.

[zachleat]

@pauleveritt just to be clear you might have a path forward right now using bundleExportKey from eleventy-plugin-bundle, a special feature for the 11ty.js family of JavaScript template types. I added some examples here: https://github.com/11ty/eleventy-plugin-bundle/issues/28

[pauleveritt]

@zachleat It turns out the issue is with addContent and this.page.url. Thanks to Khalid for spotting it. We have a decent workaround and could likely generalize for all registered bundles.

What you linked to won't work for us, because the TSX subcomponent isn't an 11ty template.

[zachleat]

I would note that the bundle shortcodes (via the addContent function) do work without this.page.url if you pass it in explicitly: https://github.com/11ty/eleventy-plugin-bundle/blob/1997df2b7e0dcb3f0959d68e7718dc23b308548f/eleventy.bundleManagers.js#L48

[pauleveritt]

Yep. In our repo we have a commented out line that shows it explicitly passed in.

[zachleat]

Related https://github.com/11ty/eleventy/issues/3355

[zachleat]

Temporary docs preview URL deploying here: https://11ty-website-git-v3-11ty.vercel.app/docs/languages/custom/#access-to-existing-filters-and-shortcodes