search

wc-duck / datalibrary

Open Source Data Library for data serialization.
Other
42 stars 8 forks source link

The reflection API is missing information about default values #161

Open Tisten opened 1 year ago

Tisten commented 1 year ago

There is no convenient way to get default values with reflection today. The only way I know of is to pass in a (semi-) empty json object which then will write a packed binary blob with the default values of a whole struct. I would like to be able to access default values per member in a struct through reflection.

wc-duck commented 1 year ago

Hmmm... that is quite a problem yes. I guess you want to access it for some editor or similar?

I honestly have no real good idea right now. If it was only for pod-types it feels easy, but for full structs and arrays. That require quite the big api.

Could something be done with a function that just "patches" an instance with just default-values? I.e. creates a fully default:ed struct somehow?

Tisten commented 1 year ago

It is for writing generative tools/compilers, yes.

I think adding access to the "per member" defaults, and "per type" default instances which are already available in memory would be the best start.

But for the cases which those doesn't cover then having a "fill instance with default values" might work as a complement. I'm not sure how that best should treat pointers and arrays though.

@ElementalAndy has the actual use case so he might have a more accurate opinion?

Tisten commented 1 year ago

I had a closer look it this now, and there is only default values for members. The members can have data with indirections (arrays, strings and pointers) as you say, and that complicates things. For the meta data I have the same issue. I would like to avoid putting the requirement of patching data like default values and meta data on the user's shoulders. My "solution" was to keep that data with pointers already patched, so when the user asks for meta data it is already ready to use. It unfortunately also means that the data isn't relocatable any longer, those instances either need to me in memory that doesn't move around, or need to be re-patched when moving them.

Having default value instances in ready-patched state in memory works fine. I've made the change locally and all tests are green, including some extra "read libraries as text" tests and tests which ping pong type libraries between stored and loaded a few times, and of cause tests which use the newly exposed meta data.

But I am pondering about which strategy is the best to avoid relocations. I used one allocation per meta data instance, and I could do the same for default data, but it does sum up to quite many small allocations. Putting all POD allocations in a batch and only allocating unique memory for instances with pointers does reduce that a lot for my data (of cause it depends on your types). Allocating with a data structure like a hive/colony/slab/"one allocation per loaded library" would also reduce the number of allocations while keeping the objects in place. And a more involved strategy is to have a state variable in the context, so when the reflection api is used the first time we patch all pointers and the context is in "patched-mode", and if a new type library is loaded then all pointers are first "reverse-patched" and put in relocatable mode again.

What is your preferred strategy?