search

nlf / protobuf.js

A super rough, super simple, protocol buffer encoding library
24 stars 7 forks source link

Streaming interface #26

Open missinglink opened 10 years ago

missinglink commented 10 years ago

More of a question than an issue...

Is it possible to use protobuf.js with a streaming interface, to; for example, stream in the 22GB open street maps db dump, map the data and pipe it elsewhere?

nlf commented 10 years ago

A proper streaming interface to protobuf.js has been on my todo list for ages.. I'll see what I can do.

missinglink commented 10 years ago

I'd be happy to dedicate some time to this feature, maybe we can start a feature branch for it's development?

nlf commented 10 years ago

Just created a streams branch, if you want to get a start on it and send a PR against that branch we'll work together a bit.

I'm envisioning this as just being two streams, an encode and a decode, each one accepts proto2json output in its constructor and does its thing. The encoder will stream raw bytes, the decoder I'm leaning toward an objectMode stream where each data event is a single key and its value.. But I'm not really set on that. Thoughts?