search

microsoft / vscode-textmate

A library that helps tokenize text using Text Mate grammars.
MIT License
561 stars 108 forks source link

`ruleStack.clone()` doesn't actually clone #228

Open RedCMD opened 4 months ago

RedCMD commented 4 months ago

The clone() function on StateStack doesn't actually clone the object, but instead just returns the original object (reference) meaning, tokenizeLine() will still modify the ruleStack that I was given

https://github.com/microsoft/vscode-textmate/blob/09effd8b7429b71010e0fa34ea2e16e622692946/src/main.ts#L262

Problem is that the clone implementation doesn't actually do any cloning https://github.com/microsoft/vscode-textmate/blob/09effd8b7429b71010e0fa34ea2e16e622692946/src/grammar/grammar.ts#L732-L734 instead something like structuredClone() should be used or use the clone function provided in the utils.ts file https://github.com/microsoft/vscode-textmate/blob/09effd8b7429b71010e0fa34ea2e16e622692946/src/utils.ts#L7-L9

Found this out when the _enterPos and _anchorPos kept getting modified externally

alexdima commented 3 weeks ago

This is intentional and is a technique to save memory.

The _enterPos and _anchorPos fields are used for temporary storage and they only make sense during tokenization of a line. No external user of the library should make any assumption about those two fields. Then, clone() returns this because all other fields are immutable. This is to save memory.