[WP] Add Token Filter to core

[philsward]

Description of the need

This is a formal request to have Token Filter added to core as a text format filter.

Both Drupal and Wordpress work on a token system in the backend, however Drupal nor Backdrop make it easy to expose the tokens to the front-end out of the box. Wordpress (from my understanding) does expose their tokens to the frontend without any additional plugins to accomplish it.

Proposed solution

Token Filter is one of my go-to modules that needs installed, just to be able to use the date token for a website copyright. It is rare that I use tokens in other places on sites (depending on the site), however the fact that none of the tokens are exposed to the frontend without a contrib to accomplish it, makes it difficult for new users to know that tokens exist.

Alternatives that have been considered

The only current alternative is to use contrib.

Another alternative is to expose a small, pre-defined set of tokens instead of all tokens. Contrib could be used to expose any additional tokens that aren't exposed by core.

Is there a Drupal or Backdrop contributed module that accomplishes this?

https://github.com/backdrop-contrib/token_filter

Additional information

I'm not big on bloating core, however this is one aspect of Drupal (and now Backdrop) that I have always pondered: "Why do tokens exist in core if core doesn't do anything to easily expose them?" I'm all for tokens being a part of core, but I'm also all for the ability to use the tokens through core.

[klonos]

https://backdropcms.org/project/usage/token_filter currently reports 180 sites using the contrib module, which is ~10% of Backdrop sites out there.

Having said that, I see how automating the copyright date is a very common use case for sites, so this might be another case of "people don't use contrib modules they don't know they exist". In Drupal, a site builder would perhaps use the php filter to accomplish this, but we've removed that in Backdrop.

Code-wise, the entire functionality is provided by ~100 lines of code, so it's not a complex module.

As an example, https://blog.templatetoaster.com/auto-update-copyright-year lists various ways to have the copyright date at the footer of a WordPress site auto-update dynamically, but most of the methods mentioned at the beginning of the article require coding. One of the methods though allows to add <span>[wpsos_year]</span> to a text field, which seems more site-builder-friendly. The [wpsos_year] token is provided by an additional WP plugin, but the ability to use such tokens (WP calls these "shortcodes") in text fields is built into WordPress core:

• https://wordpress.com/support/shortcodes

  A shortcode lets you do nifty things with very little effort. Think of a shortcode as a shortcut to add features to your website that would normally require lots of complicated computer code and technical ability.

  A shortcode is written inside two square brackets. For example, the [gallery] shortcode can be used to add a photo gallery of images to any page or post.

• https://www.wpbeginner.com/wp-tutorials/how-to-add-a-shortcode-in-wordpress

  Shortcodes in WordPress are code shortcuts that help you add dynamic content in WordPress posts, pages, and sidebar widgets. They are displayed inside square brackets like this: [myshortcode].

As an advocate of making non-coder site builders' lives easier with minimal effort + also taking into account the very minimal code of the contrib module, I'm in favor of adding this feature in core.

[klonos]

...as an additional argument, I should mention that perhaps in D7/8/9 such a feature would make more sense in contrib, since the Token module is still in contrib too. However in Backdrop we have added Token in core, so we'd be basically adding value by expanding the functionality of an existing feature.

[bugfolder]

I am very much in favor of making Token Filter readily available to a wider audience by putting it in core. I use it a LOT in D7 (and have a couple of Backdrop modules that rely on it for their functionality). However, I do note that it creates some subtle opportunities for privacy leakage.

There is no fine-grained permissions for use of the token filter (as in, "you can use this token, but not that token"). If a user has permission to use tokens in a text field, then they can use any token that exists, and that could potentially allow them to bypass various privacy controls.

For example, if a user has permissions to use tokens in a comment text field, then putting the chained token [comment:node:author:mail] into a comment will print the email address of the author of the node that is being commented upon, information that would not ordinarily be accessible.

This is not to say Token Filter is inherently unsafe; just that administrators need to be aware of the holes that it creates. So perhaps this is just an issue for documentation.

[yorkshire-pudding]

I think this should be in core as (correct me if I'm wrong) token does nothing without other modules. Possible mitigations to the security considerations could be:

• flag up with a warning if enabling on a text format with roles lower than Administrator.
• Classifying the tokens to allow some form of permissions
• provide clear help text that illustrates which are the risky ones
• Allow the admin to configure which tokens should and shouldn't be allowed in each text format.

[argiepiano]

I see the benefits of Text Filter being in core. Text filter only exposes system-wide tokens that don't require context (basically [site:XXX] tokens) and tokens related to the node/comment and user contexts (basically, giving access to fields and properties of the node/comment entities currently being created or edited, and the user(s) that created them).

As discussed above, there are privacy implications. While Token Filter doesn't display a full list of available tokens, someone familiar with tokens can easily do something like [site:mail] or [comment:parent:auhor:mail]. Things become even trickier if you have Entity Token installed, as you can also traverse through reference fields to other entities.

The idea of setting some sort of permission (or perhaps permission tiers) is good, but is not trivial to implement. The current token architecture is very decentralized. Each module defines its own tokens, and its own ways to fulfill them (basically through hook_tokens and a thread of "if" checks). Nested tokens are also traversed by each module individually. Entity Tokens was an attempt to "centralize" (or abstract) this processing by using entity properties and fields, and their metadata.

All that to say: while Token Filter would be easy to incorporate in core, the idea of centralizing permissions (if done well) would be harder to implement.

[yorkshire-pudding]

Thanks @argiepiano for the helpful context - my suggestions were made fairly blindly as I tried to look in the code for the core token module but couldn't see anything so didn't know how difficult or not it would be.

I have to say that the majority of the use cases I can think of are at the site builder/admin level (e.g. copyright notice, social sharing links) where custom blocks using full html would be the default. I therefore think that some of the warning protections would enable it to be of good use to site builders but not available to editors.

The HTML filter allows setting of which html tags can be used, so something similar in the text format filter would enable the site builder/admin to enable a limited subset of tokens where there is no risk and where there is a need.

[argiepiano]

The HTML filter allows setting of which html tags can be used, so something similar in the text format filter would enable the site builder/admin to enable a limited subset of tokens where there is no risk and where there is a need.

That would actually be a great approach and not so difficult to implement, as the module itself could decide which tokens to pass through and which ones to stop.

[ghost]

makes it difficult for new users to know that tokens exist.

I don't think we want new users to know that tokens exist... Tokens are a developer tool, and Backdrop is meant to be easy for new users. I'm sure I've heard @jenlampton say something similar before.

If a user has permission to use tokens in a text field, then they can use any token that exists, and that could potentially allow them to bypass various privacy controls.

Well that's horrifying. The site email address is exposed as a token too.

just to be able to use the date token for a website copyright.

FYI: https://github.com/backdrop-contrib/copyright_notice