Allow mutable scalars

[zuiderkwast]

Hi @mikkelfj!

I found #165 where you replied:

Also, mutation is quite limited in Flatc: you can only mutate simple scalars, and perhaps scalar arrays that doesn't grow.

Mutating simple scalars is exactly what we need. Would this be acceptable to add?

Background: In a project, we've replaced our old datatypes with flatbuffers as our internal datatypes. We store them in databases and such. But since everything is immutable, we have to rewrite the whole flatbuffer whenever some field is changed, which is hurting performance. Most of the data is numbers, so if we can just mutate that, it would be a great win. When strings/arrays/etc are changed, it's fine to rewrite the buffer.

[mikkelfj]

Well, I'm not really for adding this as a feature. Aside from concerns, it is kind of half-baked with only scalars. However, it is also very easy to do with pointer casting in C without additional support because fields have a _ptr property. There is also a mutable cast defined which is used for sorting arrays in buffers after they have been created. The C cast will only work on little endian machines (which essentially all active systems are). If you want to do it right, you cast from native endian to protocol endian (pe, in praxis this means flatbuffer little endian) using the operations you see all over the place in generated code.

I can't tell you exactly how on top of my head, but it is easy to dig out.

Each integer type has a read and a write operation from and to pe, they are defined via macros in this file: https://github.com/dvidelabs/flatcc/blob/master/include/flatcc/flatcc_accessors.h

If you choose to go this way, you could consider contributing with a documentation section for you own benefit and others, or a small sample project. But I don't see it added to the core system.

[zuiderkwast]

Thanks for the answer! It seems reasonable to me. We might look into contributing documentation, because if this is documented, it means it is supported. :-)

[mikkelfj]

BTW: you need to be careful about default values. You need to force_add fields while building to be sure there is space for updating. This in turn can take up a lot more space of you have many unused fields, as is sometimes the case in database records.

If a field is assigned null as a default value, it will always be added to the buffer when written since you can write the default value null. Not writing the field at all will then leave it at its default value null, indicated by is_present being false. In that case the field cannot be updated. https://github.com/dvidelabs/flatcc#optional-fields

Therefore you might have to check the is_present property of a field. The _ptr property would most likely be null in that case.

This is also a reason why mutable fields are not that straight forward.

[mikkelfj]

Closing as won't fix.