RomanYankovsky / DelphiAST

Abstract syntax tree builder for Delphi
Mozilla Public License 2.0
271 stars 116 forks source link

Lexer does not parse Unicode characters #302

Closed kvrabec closed 4 years ago

kvrabec commented 4 years ago

Delphi allows Unicode character encoding for its charset, but Lexer does not accept Unicode characters and raises ESyntaxTreeException

Just to clarify, I personally do not prefer to use those characters, but in old source code you can find everyting =)

e.g. const DATE_YEARDAY§TIME is a valid constant in Delphi source code, but Lexer can't parse it

Link to documentation: http://docwiki.embarcadero.com/RADStudio/Rio/en/Fundamental_Syntactic_Elements_(Delphi)