Suggestion to a better lexical handling

[GoogleCodeExporter]

I do not understand the whole "ENUM" thing in this project, regarding the types 
and type names.

I believe all the "terminals" in the lagnuage's grammar, or at least the 
typenames, should be put on an input file, that will be read into a (singleton) 
set (or, a dictionary, if needed).

This will result in:
1) Better decoupling of syntax and semantics.
2) Management of possible tokens will be centered in on data file and one 
class. So things like adding Complex128 type will become as easier as it gets - 
change the data file. No rebuild. No looking for other surprising places to add 
it there too.
3) Much shorter code: Simple set/dictionary access instead of the manual 
130-line-long 
  "if (TokenType.UINT8.equals(value)) { type = TokenType.UINT8; } 
   else if { ... "
4) Complexity of O(1) hash table search, or maybe O(logn) RB tree - instead of 
current O(n) search.

The only downside I see is the necessity to open and read another file upon 
initialization, which I believe is a minor one.

Original issue reported on code.google.com by elaz...@gmail.com on 28 Jun 2012 at 6:34