fix the custom lexer to be able to handle the whitespace at the end of the file

[paulvi]

extract as separate issue from #19 link to file https://github.com/Nodeclipse/coffeescript-eclipse/blob/master/csep/src/com/aptana/editor/coffee/parsing/lexer/CoffeeScanner.java

@cdietrich said:

as said before. we need someone that can fix the custom lexer to be able to handle the whitespace at the end of the file. for the rest a regeneration should work.

[paulvi]

Note: I wonder if there should be replaceAll ...

https://github.com/Nodeclipse/coffeescript-eclipse/blob/master/csep/src/com/aptana/editor/coffee/parsing/lexer/CoffeeScanner.java#L362

[cdietrich]

In general the whitespace handling has to be fixed. the positions in the real file and in the lexer have to match.

[cdietrich]

you can have a look at my fork to see what i have done so far. i cannot verify if my work did any damage to the correctness of the parser nor that it solves the problem in all cases.

https://github.com/cdietrich/coffeescript-eclipse/blob/master/csep/src/com/aptana/editor/coffee/parsing/lexer/CoffeeScanner.java