erg-lang / erg

A statically typed language compatible with Python
http://erg-lang.org
Apache License 2.0
2.69k stars 55 forks source link

Updated and added testing of tokenize by file input #156

Closed GreasySlug closed 2 years ago

GreasySlug commented 2 years ago

Related #80.

Tokenize testing was in progress, so updates and additions have been made.

As a result, advanced succeeds, but basic fails.

Referring to the following, ... is Spread

https://github.com/erg-lang/erg/blob/870f88e14d47b0cd57c11992ec45dd94175c7057/compiler/erg_parser/token.rs#L156-L157

... will be tokenized to EllipsisLit

a, _, ...b = five_elem_tuple

I would like you to determine if the execution results are correct or if the comment is correct.

Also, I have created this as a draft, since I want you to determine if other files should be tested as well.

Changes proposed in this PR:

9/14 update

@mtshiba

mtshiba commented 2 years ago

Thank you. Some of the grammar was outdated, so I corrected.