search

alex / rply

An attempt to port David Beazley's PLY to RPython, and give it a cooler API.
BSD 3-Clause "New" or "Revised" License
381 stars 60 forks source link

LexerIterator for LexerStream #7

Closed lucian1900 closed 11 years ago

lucian1900 commented 11 years ago

Changing LexerStream's next to raise StopIteration would have broken API, so instead I made its __iter__ wrap it in a LexerIterator.

alex commented 11 years ago

Rather than a whole new class, let's just make LexerStream into an iterator. SO change the existing next() methods to raise StopIteration, and make the Parser respect that (it should also handle None for now).

lucian1900 commented 11 years ago

If you know that no one uses the lexer directly, sure, will do.

alex commented 11 years ago

The only thing that uses the lexer directly is the parser and tests.

lucian1900 commented 11 years ago

I meant any uses of rply that might be using the lexer directly. I suppose it's not particularly popular yet :)

alex commented 11 years ago

The lexer is still relatively new, I'm fine changing the API.

lucian1900 commented 11 years ago

Ah, I seem to have broken the RPython version.

alex commented 11 years ago

It's actually just that test I believe.

lucian1900 commented 11 years ago

Ah, so the problem might be pytest's raises(StopIteration)?

[edit] I was looking at the wrong test.

alex commented 11 years ago

Well, the problem is you haven't changed the test_ztranslation.py file at all

alex commented 11 years ago

The tests which do stuff like list(iter(lexer.lex, None)) should be changed to just use list() directly on the lexer stream.

lucian1900 commented 11 years ago

Do you mean the ones using FakeLexer? They do look like they can be trivially replaced with just the list passed to them.

alex commented 11 years ago

Those, and the lexer tests.