This pr mainly focuses on extending existing script minification functionality (or remove formatting as it is called) with an ability to optionally removing comments. Removing comment tokens is not harmful as those are removed from the parse tree anyways. Along the way, i found few bugs in existing implementation of remove formatting that adds extra spaces instead of removing those like below
I also found a bug in TexlLexer that computes overlapping spans of tokens from LexComment() when it tries to consume preceding whitespaces of the first comment in the formula. This was supposed to be fix last year when this was addressed during semantic tokens work but seems like an edge case was missed.
I am keeping all the bug fixes out of scope for now since these are very low priority and remove formatting is not that widely used.
This pr mainly focuses on extending existing script minification functionality (or remove formatting as it is called) with an ability to optionally removing comments. Removing comment tokens is not harmful as those are removed from the parse tree anyways. Along the way, i found few bugs in existing implementation of remove formatting that adds extra spaces instead of removing those like below
I also found a bug in TexlLexer that computes overlapping spans of tokens from LexComment() when it tries to consume preceding whitespaces of the first comment in the formula. This was supposed to be fix last year when this was addressed during semantic tokens work but seems like an edge case was missed.
I am keeping all the bug fixes out of scope for now since these are very low priority and remove formatting is not that widely used.