Problem: how do we supply data signature information

[sphaero]

I'm suggesting to follow the struct module of python https://docs.python.org/2/library/struct.html#format-characters.

This means a vec3f will have a signature as: "3f" or "fff"

Then what do we do with dynamic data, like dictionaries or lists? You can see the 'j' character is not used. We can use the "j" character to represent a JSON string. Internally it would be handled as a "s" or "p" but we parse it before returning it?

Other serialisation formats like JSON we could possibly handle as well but let's focus on 1 first.

I think we can handle all use cases that way. Only what is inside the JSON data remains a mystery. Is that a problem?

[fieldOfView]

s would be a 0 terminated string?

I don't think 'j' would be necessary to designate a JSON string. It is a data signature, not a value signature. A json string is a string, just like a float that is part of a 4x4 matrix is still a float. NB: I agree that we need a register_object() alongside register_string() in zocp.py.

[fieldOfView]

One concern is that this decision could tie us into python. Would it be a good idea to discuss a subset of the full functionality of struct.pack/struct.unpack, so it will be easier to create a c++ equivalent? From a cursory look I cannot find a ready-to-use c++ implementation.

Eg: Do we need shorts, ints, longs, and long longs, or can we decide on only using signed longs? Would it be a good idea to only support "fff" (not "3f")?

[sphaero]

Indeed 's' must be a null terminated string otherwise you need to supply the length of the string. You are right about the 'j'. I'm just thinking their will be other use cases where you need some more verbose serialisation like JSON. If you don't provide such method the user will and it will be different every time. Not sure if that's a problem though.

Btw we might be supplying a typeHint as well which could be 'json'?

This doesn't tie us into python. C/C++ doesn't provide such functionality so you are free to do as you like. Based on the signature string you can generate a struct mirroring the data.Printf is the easiest example for such functionality. Zyre implements something like this as well.

https://github.com/zeromq/czmq/blob/master/src/zsock.c#L537

We could just stick with how zeromq is doing it. But they have a slightly different notion of data sent. They are referring to frames.

[sphaero]

Just got quite from a presentation about serialisation: Manual method is still the fastest, protobuf is 20% slower and boost is 30% slower

Manual is what we do :)