Optionally skip whitespace trimming in tokenizer

trivago / melody

Melody is a library for building JavaScript web applications.

https://melody.js.org

Apache License 2.0

215 stars 39 forks source link

Optionally skip whitespace trimming in tokenizer #139

Closed twbartel closed 4 years ago

twbartel commented 4 years ago

What changed in this PR:

TokenStream, by default, optimizes some whitespace away (e.g., around Twig comments), no matter whether ignoreWhitespace is set or not. This PR skips this whitespace trimming when ignoreWhitespace is set to false.

A test and a tiny bit of boyscouting is added, too.