search

alex / rply

An attempt to port David Beazley's PLY to RPython, and give it a cooler API.
BSD 3-Clause "New" or "Revised" License
381 stars 60 forks source link

Weird SourcePosition for \n characters #31

Closed djc closed 10 years ago

djc commented 10 years ago

It seems that \n characters get the line number for the next line, but a column number that corresponds to the last column seen on the previous line, plus 1. I.e.:

Token('NAME', 'def') 1 1
Token('NAME', 'main') 1 5
Token('LPAR', '(') 1 9
Token('RPAR', ')') 1 10
Token('NL', '\n') 2 11
Token('TABS', '\t') 2 1
Token('NAME', 'pass') 2 2

I'd argue that it should either be the last character of the previous line (e.g. 1, 11 in this example) or the first character of the next line (that would be 2, 1 here), but not some weird amalgam of both.

alex commented 10 years ago

Ugh agreed, I think it should probably be the last character on the line, do you agree?

djc commented 10 years ago

Yeah, I think the principle of least astonishment would also agree.