Right now, Tokens include a type and a Range. We include Range so that two adjacent tokens don't get merged together.
But this is a problem: if you start typing, and typing attributes (AttributedRope.Attributes) include a Token, the range of the span and the range of the token won't be the same.
This isn't a huge problem because highlighting gets rerun right after, but it's definitely odd, and going to trip me up at some point.
Possible solutions:
Make Token non-equatable. This would make two AttributedRope.Attributes that contain tokens be non-equatable by definition, which would prevent them from merging, which I think is what we want.
What happens when we're typing? First pass, before re-running our highlighter, the thing we'd want is to make the token cover the newly typed character. Perhaps that's why it's currently equatable?
Stop merging equatable spans and do the merging during the runs.
Have a separate Spans for holding tokens and make tokens non-equatable
Don't use tokens at all. Instead, just apply formatting during highlighting.
Right now, Tokens include a type and a Range. We include Range so that two adjacent tokens don't get merged together.
But this is a problem: if you start typing, and typing attributes (AttributedRope.Attributes) include a Token, the range of the span and the range of the token won't be the same.
This isn't a huge problem because highlighting gets rerun right after, but it's definitely odd, and going to trip me up at some point.
Possible solutions: