Implement BSON serialisation

bhftbootcamp / Serde.jl

Serde is a Julia library for (de)serializing data to/from various formats. The library offers a simple and concise API for defining custom (de)serialization behavior for user-defined types

Apache License 2.0

31 stars 7 forks source link

Implement BSON serialisation #19

Open dmitrii-doronin opened 3 months ago

dmitrii-doronin commented 3 months ago

Implement BSON serialisation

Description

BSON serialisation is currently marked as 'planned' in our project. This feature is crucial for users who work with YAML data formats and expect our tool to support serialisation to BSON

Tasks

[ ] Select BSON parsing libraries that fit our codebase and evaluate their styles and compatibility
[ ] Test library performance, focusing on speed, memory, error handling, and scalability
[ ] Craft an BSON serialisation interface that matches our system's design and code standards
[ ] Build the BSON serialisation logic with an emphasis on maintainability and efficiency
[ ] Develop a suite of tests for BSON serialisation to ensure robustness
[ ] Update our docs to detail the BSON serialisation process, use cases, and API changes

NeroBlackstone commented 2 months ago

One annoying thing is that BSON.jl doesn't actually support bson(io::IO; kws...), and outputs to a file by default, otherwise this feature would be very easy to implement.

help?> bson
search: bson BSON SubstitutionString AbstractIrrational AbstractFloat BoundsError

  No documentation found.

  BSON.bson is a Function.

  # 3 methods for generic function "bson" from BSON:
   [1] bson(path::String; kws...)
       @ ~/.julia/packages/BSON/mzJoC/src/write.jl:89
   [2] bson(path::String, doc::AbstractDict)
       @ ~/.julia/packages/BSON/mzJoC/src/write.jl:87
   [3] bson(io::IO, doc::AbstractDict)
       @ ~/.julia/packages/BSON/mzJoC/src/write.jl:85

A easy workaround is convert kws... to Dict{Symbol, Any} by Dict(kws) and invoke bson(io::IO, doc::AbstractDict).

I also want to ask, currently we have no implementation of outputting binary format. Should we just output Vector{UInt8} like other existing implementations, or provide an option to output to a file?

NeroBlackstone commented 2 months ago

It would be nice if you could provide some use cases for serialization and deserialization, sounds like converting yaml to bson? I'm a little confused.

dmitrii-doronin commented 2 months ago

Hi, again! That's what I had in mind. Just have some sort of a workaround to work with BSON.jl package for now. As for use cases, the most common is web development, where you want to send data more efficiently or work with binary data formats, so streaming is to a file is not really needed at the current stage so working with Vector{UInt8} and a print function should suffice.