TI-Toolkit / tokens

TI-BASIC token information XMLs for inclusion in other projects
8 stars 0 forks source link

Better two-byte token handling #13

Closed rpitasky closed 1 year ago

rpitasky commented 1 year ago

I had an idea that I was going to put in #12 but because that has already ballooned in scope, I'm putting it here instead. These changes, if we choose to go forward with them, would most likely be part of that reworking.


The more I work with this repository the more dissatisfied I am with using nested <byte> tags for two byte tokens.

Here's an abstracted view of the current structure:

<tokens>
    <byte value="">
        <token></token>
    </byte>
    <byte value="">
        <token></token>
        <token></token>
    </byte>
    <byte value="">
        <byte value="">
            <token></token>
        </byte>
    </byte>
</tokens>

I propose changing byte in the first byte of the two-byte case to two-byte (this name I'm not sure about), and every other occurrence of byte to token. What is currently called token should really never have been called token and instead should be version:

<tokens>
    <token value="">
        <version></version>
    </token>
    <token value="">
        <version></version>
        <version></version>
    </token>
    <two-byte value="">
        <token value="">
            <version></version>
        </token>
    </two-byte>
</tokens>

I think this communicates what we want much better.