search

NightlyCommit / twing

First-class Twig engine for Node.js
BSD 2-Clause "Simplified" License
199 stars 23 forks source link

Add a way to parse templates in a "loose" way #541

Closed ericmorand closed 3 years ago

ericmorand commented 3 years ago

Twig langage unfortunately permits to customize the core specification of the language by adding global functions, filters and tests. Even though it is strongly recommended to not take advantage of this design oversight - and provide customization at runtime through the context passed when rendering a template, a lot of frameworks have unfortunately based their templating engine on the usage of those global "callables": Drupal, Wordpress, Symfony...

Currently, Twing parser throws an error if an unknown "callable" is encountered while parsing the template, making impossible to parse templates using custom callables without knowing before-hand the complete list of those callables.

To overcome the issue, we need to allow the parser to function in a "loose" way where it doesn't fail when encountering an unknown callable.

Discussed in https://github.com/NightlyCommit/twing/discussions/540

Originally posted by **adambrgmn** October 3, 2021 Hey! First, thanks for this very powerful module @ericmorand! I have a question that you maybe know the answer too already or might be able to direct me towards. I'm building an internal tool (maybe public in the future) where we want to be able to extract translations from twig templates within WordPress projects. This means finding references to the WordPress translation methods (`__`, `_n` etc.) in `.twig` files. Previously we've used a regex, but it has proven not to be very safe. So I'm looking into AST parsing and walking the tree to find calls to certain functions. But the thing is, I want to make this tool independent of what each project uses, therefore I have no idea which functions and other extensions the current project has enabled. Then if I try to parse a file errors will be thrown for all unknown functions. Which is not very useful. And it would require too much configuration to repeat all defined functions/extensions etc. per project. This is a sample of the code I have that will throw errors: ```ts let loader = new TwingLoaderNull(); let twing = new TwingEnvironment(loader); let source = new TwingSource('{{ __("Hello", "domain") }} world!', ''); let stream = twing.tokenize(source); let ast = twing.parse(stream); ``` I also found a reference in an issue to you project [`twig-to-phtml`](https://github.com/NightlyCommit/twig-to-phtml/blob/master/src/lib/Transpiler.ts). But as far as I understand it requires you to register function names before running `transpile`? Is it possible today to parse source code without throwing errors for unknown functions? Or could you maybe direct me towards something that would lead me towards it? Maybe I need to write my own [`TwingParser`](https://github.com/NightlyCommit/twing/blob/master/src/lib/parser.ts)?
ericmorand commented 3 years ago

@adambrgmn, you should now be able to parse a template in a "loose" way - that is without the parser checking if functions, filters or tests are registered - by passing {strict: false} as second parameter to TwingParser constructor.