dvidelabs / flatcc

FlatBuffers Compiler and Library in C for C
Apache License 2.0
653 stars 189 forks source link

How to distinguish multiple tables #141

Closed snikeguo closed 4 years ago

snikeguo commented 4 years ago

My solution is: enum PackageType{Type1,Type2} table Type1Table{...} table Type2Table{...}

table PackageTable { type:PackageType; data:[uint8];//It may be type1table or type2table }

//Serialization: PackageTable_Type_add(Type1); var buf=Type1TableSerialization(); PackageTable_data_create(builder,buf,size);

//Deserialization: var type =PackageTable_Type_get(rec_buf); if(type==type1) { Type1TableDeserialization(rec_buf,offset); } Here is a question, how to convert a sub-table(type1table,type2table) into a u8 array? (var buf=Type1TableSerialization();)

snikeguo commented 4 years ago

Do you have a better solution?

mikkelfj commented 4 years ago

Without going into a lot of details (you can see the monster_test project for examples), what you are doing is to nest one buffer as a byte array in another buffer. There is official support for this concept by adding a nested buffer attribute in the schema. It works much like you suggest, but there are tricky details about ensuring the nested buffer is properly aligned so you can read it without copying into aligned memory.

However, this is not generally the most convenient way to deal with the problem, but it depends on the exact circumstances. Often people need to send one of several different messages. This can be done by creating a union of tables. Instead of the type field you manually add, a union has an implied type field and a data field holding a table (or a string or a struct). I advise you to read more about unions this both in the official flatbuffers project, notably the Monster sample, and in flatcc samples and documentation.

A third option is to skip the parent table (or envelope table) and instead use the file_type in the buffer header to tell the difference. FlatCC has support for assigned a unique type for each table based on a hash of its name. This is not directly supported by other languages, but these languages can read the file type so you should still be able to do the test. The advantage is less overhead in size and speed, and in that the different tables do not need to belong to the same schema, but most of the time it is simpler to use union.

Yet another approach is to keep the type field you have but add both tables as fields as well, instead of the byte array. You then only write to the one field that you use. This is seen as a dirty hack, and if you have many different tables, the internal "vtable" that tells which fields are in use, can take up space. However, in simple cases this can be simpler than adding a union, for example if you have a table representing a shipping address and sometimes have extra fields such as a secondary phone number.

mikkelfj commented 4 years ago

Based on the above information I can point you to more detailed information once you decide what is best for you.

snikeguo commented 4 years ago

My application scenario is USB, serial port, CAN (such as: J1939 / ISO15765). In order to cooperate with automatic code generation, I finally wrote a small serialization framework myself https://github.com/snikeguo/EmbedXrpc/blob/master/EmbedXrpcSerialization/EmbedXrpcSerialization.h

mikkelfj commented 4 years ago

You can do that, and it can be a steep learning curve to use a framework such as flatbuffers, but in my experience it pays off in the long term because you have a schema and it is easy to add or change the data being exchanged in a robust manner. If you want something simpler there is also msgpack.

In your use case I would recommend using a union to distinguish tables if you decide to use flatbuffers. An example can be found here, although it is not used for different messages. https://github.com/dvidelabs/flatcc/blob/401cee0d0840dee8ab2c257af9e928049a17b05a/samples/monster/monster.c#L77

More about the sample project here: https://google.github.io/flatbuffers/flatbuffers_guide_tutorial.html