google / flatbuffers

FlatBuffers: Memory Efficient Serialization Library
https://flatbuffers.dev/
Apache License 2.0
23.28k stars 3.25k forks source link

How do I build a vector of tables? #4151

Closed voznesenskym closed 7 years ago

voznesenskym commented 7 years ago

I am working on a C++ project using flatbuffers.

My model is:

namespace msd.models;

table Node {
key:string;
size:double = 0;
next:string;
}

table Nodes {
nodes:[Node];
}

root_type Nodes;

I build nodes into an instance of fbb:

Cache::Node Cache::Node::deserialize(const models::Node *modelNode, flatbuffers::FlatBufferBuilder& fbb){
    Node node = Node();
    node.writeKey(modelNode->key()->str());
    node.nextKey = modelNode->next()->str();
    node.size = modelNode->size();
    return node;
};

However, when I go to build table Nodes, I have 2 choices:

Use the same fbb instance (bad, because then the buffer contains both my built nodes and the new vector, Nodes)

or

Make a new fbb instance, which crashes because the offsets of the original fbb instance are now essentially meaningless. How do I build a new buffer? how do I build:

table Nodes { nodes:[Node]; }

if I can make a vector by way of:

    std::vector<flatbuffers::Offset<models::Node>> serializedNodes;

    for (auto it = nodeMap.begin(); it != nodeMap.end(); ++it ) {
        flatbuffers::Offset<models::Node> node = it->second->serialize();
        serializedNodes.push_back(node);
    }
aardappel commented 7 years ago

I'm not entirely sure what you're trying to do.. you say, "I build nodes into an instance of fbb", but then the code doesn't use fbb but instead appears to be copying data from a FlatBuffer into a C++ data structure.. which I am not sure how that relates to creating a vector of tables.

Your loop below there should work, assuming serialize returns the result of a call to CreateNode. Both the individual nodes and and the Nodes table should indeed go into the same FlatBufferBuilder to be able to refer to eachother.

If I missed what you're trying to do, try posting an as complete as possible code snippet, and I'll point out where it is going wrong, if anything.

voznesenskym commented 7 years ago

I found the issue :) I was calling fbb.Finish() in my serialize step. The solution is:

flatbuffers::Offset<models::Node> Cache::Node::serialize() {
    models::NodeBuilder builder = models::NodeBuilder({fbb});
    builder.add_key(flatBufferKey);
    if (next != nullptr) {
        builder.add_next(next->flatBufferKey);
    }
    builder.add_size(size);
    flatbuffers::Offset<models::Node> node = builder.Finish();
    return node;
}

Called from:

    std::vector<flatbuffers::Offset<models::Node>> serializedNodes;

    for (auto it = nodeMap.begin(); it != nodeMap.end(); ++it ) {
        flatbuffers::Offset<models::Node> node = it->second->serialize();
        serializedNodes.push_back(node);
    }

    auto vectorOffset = fbb.CreateVector(serializedNodes);

    models::NodesBuilder builder = models::NodesBuilder(fbb);
    builder.add_nodes(vectorOffset);
    flatbuffers::Offset<models::Nodes> finishedOffset = builder.Finish();

    fbb.Finish(finishedOffset);