Goals of `jsonparse`

[mgirlich]

• faster than jsonlite: 1) use a faster JSON parser and 2) use C++ to parse into the correct structure instead of R
• vectorised parser: jsonlite isn't vectorised but pastes input together. When working with JSON columns this is very annoying and slow.
• better handling of NA: jsonlite always produces an error message for NA input. The user should be able to control this (especially in a vectorised version).
• automatically detect output format like jsonlite: this is one of the strengths of jsonlite
• more robust than jsonlite: there is currently no parser that allows the user to specify how the output of the parsing should look like. Allowing the user to do so is more robust and much faster
• support JSON path to extract nested fields from JSON: often enough I only need a small subset of the JSON but first have to parse the whole JSON and then extract the required fields via purrr.
• good handling for big integers: in jsonlite there is only a hidden option and it is not very robust as sometimes your numbers might all be small enough to actually fit into an integer.
• option to return JSON: not sure how common it is but in some situations I didn't really want to parse a field but wanted to get it back as JSON.
• maybe support some operations of the jsontools package: e.g.
  • unwrapping arrays (of scalars) with json_unnest_longer()
  • unwrapping objects with json_unnest_wider(); probably this is by now possible with mutate() + user specified structure for parsing
  • json_hoist(); probably the same as for json_unnest_wider()

[krlmlr]

Would RcppSimdJson be faster than jsonlite already, perhaps fast enough for most use cases (even if we don't support parsing only subsets)? Could we achieve laziness (and speedup for partial parsing) with ALTREP?

https://cran.r-project.org/web/packages/RcppSimdJson/index.html

[mgirlich]

I know about RcppSimdJson and thought resp. think about contributing to it. At least for now I wanted to create a new package because:

• to contribute I need(ed) to understand more C++ (before I started working on this I had barely any knowledge)
• to contribute I needed to understand more about simdjson
• I really like to be able to parse according to a specification
• I would like to use the ON DEMAND API from simdjson
• less coordination and simpler toying around for now

I am currently finishing my work on a first prototype. Would you be interested (and have time) to contribute a little to this project?

[krlmlr]

Thanks. I'm happy to take a look, I can't commit substantial time to that project though.

On the other hand, I wonder what it takes to bring a green "stable" badge to tibblify. I wonder if e.g. we could reuse the existing tests and infrastructure in tibblify to avoid reinventing substantial parts of the wheel in the new project.