Improve charlist combinator

mracos commented 5 years ago

Tried to explain what was done in the commit messages, if it's not that clear please ping so I update this PR description. 😄

Before

After

Things that I already mapped on #6 and this PR address:

[X] Fix the combinator for strings for some cases (for example, the "V +" is being tokenized as something like [:punctuation, :variable, :whitespace, :operator]) and it should be a single unit
[X] Fix the combinator for the interpolation characters, that as far as I can tell are not being tokenized at all
[X] Add the string token to the binary combinator (<< >>)

I've been based all my changes on the "semi-official" erlang/spec, mostly the Grammar Appendix.

josevalim commented 5 years ago

:heart: :green_heart: :blue_heart: :yellow_heart: :purple_heart:

tmbb commented 5 years ago

Good job! Sorry for having so "hands-off" here

elixir-makeup / makeup_erlang