Fixing regex so that ACE tokenizer can work properly

Ezhil-Language-Foundation / Ezhil-Lang

எழில் - ஒரு தமிழ் நிரலாக்க மொழி; தமிழ் மாணவர்களுக்கு இது முதல்முறை கணிப்பொறி நிரல் ஏழுத உதவும் (Ezhil, is a fun Tamil programming language for K-12).

http://ezhillang.org/

GNU General Public License v3.0

170 stars 46 forks source link

Fixing regex so that ACE tokenizer can work properly #51

Closed marathonfail closed 11 years ago

marathonfail commented 11 years ago

Earlier not all Ezhil programs used to work with ACE as the regex was incomplete leaving the tokenizer to hang in the middle. Fixed it through this code change.