jcrist / msgspec

A fast serialization and validation library, with builtin support for JSON, MessagePack, YAML, and TOML
https://jcristharif.com/msgspec/
BSD 3-Clause "New" or "Revised" License
2.45k stars 76 forks source link

ormsgpack benchmark comparison #608

Closed LockedThread closed 11 months ago

LockedThread commented 11 months ago

Question

Are there benchmarks against ormsgpack? I currently use ormsgpack for work in combination with pydantic but the speed degredation of pydantic for validating our extremely large objects is becoming a bottleneck. I am looking for alternatives?

jcrist commented 11 months ago

Thanks for opening this, I hadn't heard of ormsgpack before. I've added it to our benchmarks, see the msgpack benchmarks here.

A brief summary:

This may seem counterintuitive - how can msgspec be faster at decoding AND validating than decoding alone? This is due to two reasons:


Anyway, the gist is that

I originally wrote msgspec specifically as an efficient pydantic-like thing with a high-performance msgpack decoder, so your use case sounds like the ideal case for this library. If you decide to try it out, please let me know if you run into any rough edges.