cenotelie / hime

Apache License 2.0
27 stars 4 forks source link

Terminal not recognized #72

Closed SadE54 closed 3 years ago

SadE54 commented 3 years ago

Hi,

I have a grammar like that terminals { WHITE_SPACE -> uc{Zs} | U+0009 | U+000B | U+000C ; SEPARATOR -> WHITE_SPACE+; IDENTIFIER -> ([1-9][0-9]+) ; STRINGIDENTIFIER -> ([a-zA-Z] [a-zA-Z0-9_]) ; } rules { property_id -> 'id'; value_id -> IDENTIFIER | '?' ; assignment_operator -> '=' ; assignment_id -> property_id assignment_operator value_id ; property_assignment -> assignment_id | assignment_width | assignment_height; command_set -> 'set' STRING_IDENTIFIER property_assignment ; }

If I type 'set test_object id = ? it's ok but if i try : set testçobject id = 123 I have the error : Unexpected token "123"; expected: IDENTIFIER, ?

I don't understand because my regex for identifier seems ok

woutersl commented 3 years ago

I tried to replicate with the following that works as expected:

grammar Test {
  options {
    Axiom = "command_set";
    Separator = "SEPARATOR";
  }
  terminals {
    WHITE_SPACE -> uc{Z} | U+0009 | U+000B | U+000C ;
    SEPARATOR -> WHITE_SPACE+;
    IDENTIFIER -> ([1-9][0-9]+) ;
    STRING_IDENTIFIER -> ([a-zA-Z_][a-zA-Z0-9_]*) ; // <- removed a space here between the two []
  }
  rules {
    property_id -> 'id';
    value_id -> IDENTIFIER | '?' ;
    assignment_operator -> '=' ;
    assignment_id -> property_id assignment_operator value_id ;
    property_assignment -> assignment_id; // <- removed the undefined variables
    command_set -> 'set' STRING_IDENTIFIER property_assignment ;
  }
}

This works correctly on set test_object id = 123. (I expect this is what was meant, and not testçobject, which cannot be matched by the STRING_IDENTIFIER rule).

SadE54 commented 3 years ago

Sorry I think I found what was wrong . I only pasted extracts of the grammar and there was another terminal placed after IDENTIFIER that probably overloaded it : INTEGER_LITERAL_DECIMAL -> ('0' | [1-9] [0-9]*) ([Uu] [Ll]? | [Ll] [Uu]? )? ; using this one instead INTEGER is working now as expected. My apologies !

woutersl commented 3 years ago

That makes sense: no problem. I'll close this ticket, feel free to open a new one, shall another issue arise.