Use NamedTuple and Enums

[ThatXliner]

This will require a major code change, I think.

[ThatXliner]

Actually, a lot of code could be using enums

[adamsol]

I want Token objects to be uncomparable with anything, because it's very easy to make a mistake and compare them directly with a string, which would always produce False and cause hard-to-find bugs. So NamedTuple is probably not a good solution. I'd wait to drop Python 3.6 support and use dataclasses from Python 3.7.

Personally, I don't see much benefit in using enums instead of constants directly in the Token class. Also, the name TokenTypeEnum is too long, maybe TokenType would be enough. Anyway, if you want to make such a change, please make sure to fix all the necessary places (tokens are also used in parser.py). Also, it would be best to avoid repeating the names, like here: https://stackoverflow.com/q/58471615/.

Actually, a lot of code could be using enums

Right, e.g. certainly a Node enum instead of the hardcoded AST node names would be useful. However, that would be a much bigger change.

[ThatXliner]

I see. So the enum idea is a good one, but I'll need to use enum.auto() as values instead.

On the Python support, why not just skip directly to 3.8?

[adamsol]

I want the code to be compatible with the versions of Python that are still officially supported: https://endoflife.date/python.

[adamsol]

Hey, are you still interested in this change?

[ThatXliner]

Not really. You can merge it if you want.

[adamsol]

Doesn't currently pass tests, so maybe I'll clean this up later.