Removed Tokenizer; created LexerInput wrapper

[CinchBlue]

I have implemented LexerInput to wrap std::istream& and also provide get(), peek() and unget(). These methods also make sure to keep track of the line, column, and position numbers. This means that this serves as a good primitive to implement the Lexer (new Tokenizer design) on, as we'll just be able to just copy the current line information stored in the struct LexInfo over into the Token.

• Token has been modified to use LexInfo. I have not done any unit tests for this because LexInfo is just a plain old data (POD) class that has no methods to speak of.

Please check that the LexerInput test cases are good -- the semantics for unget() are a bit strange, and I want your feedback on it.

Please also check for general mistakes or faulty asserts in the implementation or test case.

[CinchBlue]

I am going to close this pull request. I have done some independent work on the LexerRules and my own parser combinator functionality that I will include in a future pull request.