search

hoaproject / Compiler

The Hoa\Compiler library.
https://hoa-project.net/
453 stars 48 forks source link

Incorrect calculation of line and position in utf-8 files #78

Open SerafimArts opened 6 years ago

SerafimArts commented 6 years ago

Example code with an error on the last line:

type A {
    some(arg: String = "😺 😸 😹 😻 😼 😽 🙀 😿 😾"): Any
}

😿

Expected (line 5 and column 1):

Unrecognized token "😿" at line 5 and column 1:
😿
↑ <- arrow here

Actual (line 1 and column 89):

Unrecognized token "😿" at line 1 and column 89:
type A {
    some(arg: String = "😺 😸 😹 😻 😼 😽 🙀 😿 😾"): Any
}

😿some

                                                             ↑ <- arrow here
flip111 commented 5 years ago

@SerafimArts i might take a look at this bug, but please make it easy for me and provide runnable test code (i don't need a full hoa Test case).

It would be good to fix this bug together with #97 at the same time.

SerafimArts commented 5 years ago

Just write any character from the UTF 4bytes group to reproduce the problem.

grammar:

%skip space \s

text:


          😺 😸 😹 😻 😼 😽 🙀 😿 😾