search

wc-duck / datalibrary

Open Source Data Library for data serialization.
Other
42 stars 8 forks source link

DL duplicates data when using pointers to structs #152

Open Tisten opened 1 year ago

Tisten commented 1 year ago

I've created a test for this in https://github.com/wc-duck/datalibrary/pull/151

I could try to solve it myself, but I'm not sure how to best put a tag on some data in JSON so a pointer can refer to it. Any opinions?

lundmark commented 1 year ago

From my PoV I'd appreciate it if they were separate json-objects serialized just like the actual data would look like in memory but as json instead. Then have the structs refer to them by index or something similar?

I guess there should be a storage passed in the serialization context that handles all existing references to serialized objects to handle this?

Tisten commented 1 year ago

So all structs which someone points to will get a magic "__index" member which pointers can refer to?

lundmark commented 1 year ago

Something like that would make sense imo yes, I'm not sure what @wc-duck thinks tho?

wc-duck commented 1 year ago

hmm... in that case we could just use any name, such as "__ref_id" or similar so that if you write it by hand you can just write something that makes sense. It could always be generated as "my_type_1" or similar.

But also, this would only be needed if you reference an item that is also part of an array or struct right? Otherwise they are just put in 'subdata' : { 'item_1' : {} }

Right? Long time since I worked with the pointers? :)

Tisten commented 1 year ago

Yes, that is right. I agree that writing something sensible when authoring it by hand is a great feature. And keeping pointer payloads which aren't used as arrays or members of structs is also fine, it is one correct way of doing it, and the closest to what is there today.

For improved json readability I could also imagine it working by putting each pointer payload in the place where it is first referred to, and then all other pointers to it will just refer to the "__ref_id". But that is a matter of taste, both would work and option 1 is probably least work.