search

jcrist / msgspec

A fast serialization and validation library, with builtin support for JSON, MessagePack, YAML, and TOML
https://jcristharif.com/msgspec/
BSD 3-Clause "New" or "Revised" License
2.45k stars 76 forks source link

feat: support deserializing to memoryview #624

Closed SunDoge closed 11 months ago

SunDoge commented 11 months ago

The implementation enables zero-copy functionality when decoding msgpack by supporting deserialization directly to a memoryview. This is very useful for decoding large numpy array.

jcrist commented 11 months ago

Thanks for the PR! I've taken the liberty to finish it up - adding support for new types needs to be done in several places.

Note that zero-copy deserialization of buffers can also be handled through Ext types. When decoding an Ext, the buffer passed to ext_hook is a memoryview into the original backing message. This was intended to support decoding large buffer-backed objects (like numpy arrays) in a zero-copy fashion.