msiemens / tinydb

TinyDB is a lightweight document oriented database optimized for your happiness :)
https://tinydb.readthedocs.org
MIT License
6.76k stars 534 forks source link

Use orjson instead native json to improve performance #521

Closed gaetan1903 closed 1 year ago

gaetan1903 commented 1 year ago

orjson is a fast, correct JSON library for Python.

it's written in rust with maturin, so it is fast

A simple image to show difference of performance

WRITE DATA

save

QUERY DATA

save2

msiemens commented 1 year ago

Thanks for your contribution, @gaetan1903! Unfortunately, I'll close this pull request. It's not that your suggestion or code is bad but rather that it's part of TinyDB's design (and its selling points) that it does not have any external dependencies. However, if you turn this into a new storage backend and release it on PyPI, I'll gladly link to it in the documentation and mention the performance benefits. That way users can decide if they prefer using Python's native json module or if they'd rather install (and potentially compile) a binary dependency in order to gain a performance increase. I hope you understand the reasoning behind this! If not, feel free to ask 🙂