search

google / flatbuffers

FlatBuffers: Memory Efficient Serialization Library
https://flatbuffers.dev/
Apache License 2.0
23.28k stars 3.25k forks source link

[C++] ensure_space assert failed #5081

Closed cklsoft closed 5 years ago

cklsoft commented 5 years ago

I use flatbuffers c++ to serialize a big object, and meet an assert belowing:

flatbuffers.h:590: size_t flatbuffers::vector_downward::ensure_space(size_t): Assertion `size() < ((1ULL << (sizeof(soffset_t) * 8 - 1)) - 1)' failed.

Why such assert exist? It is that mean flatbuffers only allow FlatBufferBuilder buffer_size < 2G? So how can I avoid such error? I have to split the object to smaller parts?

cklsoft commented 5 years ago

I found this assert is casued by the fllowing: auto data_vec = fbb.CreateVector(data_buf, byte_len);

Can I change to the following? const std::vector data_vec(data_buf, data_buf + byte_len);

aardappel commented 5 years ago

Yup, FlatBuffers uses 32-bit offsets. We could offer a 64-bit version of the format, but never got around to it. See notes on FlatBuffers64 here: https://github.com/google/flatbuffers/projects/10

I'm not sure what your change is intended to do (since std::vector will not create any serialized data), but it won't get around the 2GB limit.

Your only option for now is to split up your data in smaller buffers. Or help make FlatBuffers64 a reality :)

cklsoft commented 5 years ago

Thanks for your reply:D My solution works. I will try your suggestion.