search

rust-crdt / rust-crdt

a collection of well-tested, serializable CRDTs for Rust
Apache License 2.0
1.36k stars 58 forks source link

backward- and forward-compatibility & `CmRDT`/`CvRDT` derive macros #53

Open chpio opened 4 years ago

chpio commented 4 years ago

Hi, is there already a best-practice regarding backward- and forward-compatibility for crdts? Say i want to persist some data (not a far-stretch in context of crdts) and i need to support data created by an older version of the program and also older versions of the program to read data coming from newer versions of the program. Is it possible to do that automagicaly by tagging the serialized data, so that older program versions are able to merge/apply newer version data by knowing the crdt-data-type?

Also would it be feasible to have something like CmRDT/CvRDT derive macros for custom structs (i think it would be harder to get enums working)? ProtoBuf is deserializing unknown fields into a hidden struct field, can we do that too?

Is this related to https://github.com/rust-crdt/rust-crdt/issues/51 but with "more" structured data?

davidrusu commented 4 years ago

backward and forward-compatibility for crdts

Great question, I'm surprised I haven't seen this topic come up before. I don't have a good answer for you, I could speculate on possible solutions but based on my experience building CRDT's, the edge cases will be subtle. This would be an interesting topic for some research

derive macros for custom structs

I think it might be possible yeah, I'm not deeply familiar with macros, but some annotating of struct fields with their CRDT types and we should have enough information to derive a CRDT. There might be some complications with CmRDT's because we'd need to generate the Op enum, but maybe it'll be fine.

ProtoBuf is deserializing unknown fields into a hidden struct field, can we do that too?

Can you share an example?

Is this related to #51 but with "more" structured data?

We already support recursive data types (see Map), so that should not be too much of an issue

JLerxky commented 1 year ago

I have an implementation of proc_macro. Can you take a look and see if it meets the requirements for custom CRDT structs outlined in this issue? https://github.com/JLerxky/crdts_derive