Add more detailed scopes to the tree-sitter grammar

[ambv]

I made changes to tree-sitter-python.cson to enable better highlighting. This includes:

• fixing keywords from all being passed "keyword.control" scope
• fixing numerals from being treated as language constants
• adding missing keywords (nonlocal, global, del)
• removing deprecated keywords (exec, python) which are regular functions in Python 3
• adding punctuation as its own class
• adding missing operators (**, //, @, comparisons and assignment)

Scope names

I used scope names from python.cson so that themes are automatically compatible without changes. Let me know if you'd like different names.

Builtin functions/exceptions don't work yet

The builtin functions and exceptions are a useful feature in Python code. Unfortunately, it appears that currently the scope is not applied to them. I left what I tried in the pull request but this requires some changes to actually work.

I'm not sure how to even debug this as Atom is not actually scopes with Tree Sitter enabled (it's showing the syntax tree at cursor).

Builtin functions and exceptions are rather easily generated from Python (the names are present in the builtins module, the ones that are subclasses of BaseException are exceptions, others are functions). It should be relatively easy to keep this in sync in the future.

There are no tests

I'd like to use @kevinastone's Atom-Grammar-Test for it once #254 lands. I can do that as a separate pull request later.

[maxbrunsfeld]

There are no tests. I'd like to use @kevinastone's Atom-Grammar-Test for it once #254 lands. I can do that as a separate pull request later.

We could use atom-grammar-test, but we'd have to modify it a bit to support the Tree-sitter based highlighting.

The actual parsing is tested quite exhaustively by the tree-sitter-python test suite, and tree-sitter-python.cson file in this repo is pretty declarative, so I don't think that we necessarily need additional tests at this layer, but I'd be ok adding them if you feel they'd add value.

[ambv]

General question: should I add ".python" to every scope or remove them everywhere? Currently this is not consistent, what do you think?

[maxbrunsfeld]

I was hoping to avoid having the redundant python scope everywhere, but it seemed like it was used by the bundled themes for a couple of tokens in particular, so I added just to make those scopes work. I guess we should just add it everywhere.

[khaeru]

I just noticed the tree-sitter parsers in the changelog/settings pane, and tried enabling them…but losing the distinct color for numeric constants (lots in the scientific code I work on) sent me right back.

Is there any help needed to get this merged?

[nathansobo]

This code was merged in via #288. Thanks for getting it started and sorry it took so long. ⚡️ ❤️