search

anilmaurya / fast_jsonparser

Fastest Json parser for Ruby, wrapper for simdjson
MIT License
306 stars 10 forks source link

Streaming support? #2

Closed thbar closed 4 years ago

thbar commented 4 years ago

I just discovered this gem. Congrats, this seems very useful!

I wonder if you will (or already) support streaming processing of large files, either newline delimited files, or just large files?

Thanks!

anilmaurya commented 4 years ago

Hi @thbar

This gem is wrapper over https://github.com/simdjson/simdjson, simdjson does not support streaming directly. I will check how to do it. would you like to help ?

thbar commented 4 years ago

I just had a little look at the documentation. In particular this seems interesting:

https://github.com/simdjson/simdjson/blob/master/doc/basics.md#newline-delimited-json-ndjson-and-json-lines

I do not have a strong need yet, but I will update you if the strong need comes, in which case I'll be happy to help!

anilmaurya commented 4 years ago

Hi @thbar I just release 0.3.0 with file streaming support, have a look at it.

thbar commented 4 years ago

Hi @anilmaurya - thank you for your work! Amazing. I will try it out. Closing this issue!