isagalaev / ijson

Iterative JSON parser with Pythonic interface
http://pypi.python.org/pypi/ijson/
Other
615 stars 134 forks source link

Control buffer size #31

Closed acrisci closed 9 years ago

acrisci commented 9 years ago

I have an application that reads a continuous stream of json data from a process as an infinite array. I need to read an object out of this array about once per second. Each object is about 200 characters long.

The current buffer size is 16 * 1024. This means that I don't get events until the buffer is full, which takes a very long time. I need to make the buffer size very small for my application to work correctly.

I would like to add a kwarg to ijson.parse() to control the buffer size.

pwalsh commented 9 years ago

+1

isagalaev commented 9 years ago

That is a very interesting usecase indeed. Could you make a patch for this? Should be pretty straightforward…

acrisci commented 9 years ago

Looking through the code, I found I could already do this with the yajl backend.

jmarshall9120 commented 6 years ago

2018 calling.... @acrisci , do have an example of this?

acrisci commented 6 years ago

@jmarshall9120 here is the use case I was referring to in the issue.