Closed smheidrich closed 2 years ago
Hey, I'd love to see a PR with this?
I've released a new version 1.4.0 with this to pypi, and added to the docs: https://github.com/daggaz/json-stream#custom-tokenizer
Perfect, thanks a lot! :+1: I've updated my package as well.
To make this library a bit faster, I ported the tokenizer to Rust: https://github.com/smheidrich/py-json-stream-rs-tokenizer
Currently this works by monkeypatching the Rust tokenizer over
json-stream
's own tokenizer inloader
andvisitor
, which is pretty hacky.Do you think it would make sense to give
load
andvisit
an extra parameter that allows users to pass in a custom tokenizer? Then I could use that instead of monkeypatching.