smhg / gettext-parser

Parse and compile gettext po and mo files, nothing more, nothing less
MIT License
158 stars 44 forks source link

Add gettext-parser Stream API #5

Closed ArnaudRinquin closed 9 years ago

ArnaudRinquin commented 9 years ago

Purpose: Allow .po/.mo files Stream parsing.

Includes, for .po:

Includes, for .mo:

andris9 commented 9 years ago

Generally I like the idea. I reimplemented it (see aaa83f561c45efd6e55e5088ed18831fb4f22b53) because I wanted to use native stream.Transform instead of an additional dependency. The proposed implementation was not also unicode safe – if the input chunk ends in the middle of a utf-8 sequence, it would mess up the unicode character. There was also a small bug with escape symbols because the data about it was not passed on from one _lexer run to another. Otherwise I mostly reused your ideas of how to handle this.

I'm not too sure if mo parsing should also support streaming or not as it is hard to see what advantage would it provide besides unified API. mo file is basically a memory map, so even if we parse an input string early, the corresponding translation might be on the other end of the file.

ArnaudRinquin commented 9 years ago

I am bit sad that none of my original code made it to the current version but your rework makes it a lot better. That's great.