Freedom-of-Form-Foundation / anatomy3d

A CAD tool for humanoid anatomy alterations. See the anatomy3d-blender repository for more recent work.
https://freedomofform.org/1856/3d-anatomy-project-scope-phase-1-focus-on-a-limb-joint/
GNU General Public License v2.0
7 stars 5 forks source link

Meta issue: Design and implement (protocol buffer) file format #19

Open Lathreas opened 3 years ago

Lathreas commented 3 years ago

For storing any anatomy files, we need to store the data that represents each species or character. We should decide on how exactly to store this data and implement the save functions.

It might be a good idea to store a list of objects and serialize them easily using the stl System.Text.Json to keep things easy and clear.

AdamNorberg commented 3 years ago

I strongly oppose JSON for this use because it is an incompletely-defined format (the spec is, approximately, "do what JavasScript does"), it does not allow declarative types providing consistency or tools enforcing what fields are expected, and its numeric type is always implied to be double, which cannot represent every long integer and certainly not every unsigned long integer.

I strongly prefer protocol buffers: https://developers.google.com/protocol-buffers/docs/csharptutorial for the C# version. It's a strictly written standard, it uses schemas with proper validation, it has a very compact binary representation, and it has been tuned to an extraordinary degree for encoding and decoding efficiently. Of course, I'm biased, because I work with them on a daily basis...

JSON has the nasty problem that it's easy to write something that looks like it's working but isn't. Protocol buffers are more opinionated about their contents, while being lax enough to allow fields to be added during development and correctly passed on by systems not familiar with those fields if that system is simply transferring messages, because it is actively designed to be "forwards compatible" and allow unknown fields to be transferred and reserialized.

Protocol buffer definition files run through protoc and generate native C# code. One of the benefits to this is that the C# compiler can (and will) stop you from using the wrong name or type for a field because it is a formally-defined type. The downside is that now the build process goes from "compile" to "generate, compile" but it is the nature of all nontrivial software projects to get an increasingly complicated build pipeline over time...

Lathreas commented 3 years ago

Let me take a look at protocol buffers! They look very interesting so far, and I definitely see their benefits. Especially the fact that it handles floating point numbers much better than JSON is a big plus.

Since it uses a precompiler, I wonder how well it works with Godot, though. Internally Godot uses mono to handle C# compilation, so maybe it is possible to smoothly pass the precompiler (or generator?) to Godot's current build command, but that's something we should investigate.

To be fair, I must admit I am personally not too fond of adding too many precompilation steps if we can avoid it, so let's see what we can do there.