Encoding to and decoding from buffer(memory)

[ToKiNoBug]

I found this library helpful and neat, but I am using nbt as a part of another binary format, so I hope to encode and decode from memory buffer instead of file streams. Please tell me how can I encode nbt to memory buffer, and decode a nbt from memory.

If this function is not implemented yet, I am willing to do it myself and open a pull request.

[nickelpro]

Happy to accept a PR, but I don't really understand. Any mechanism that can "read from a buffer" is going to be isomorphic to wrapping that underlying buffer type in an istream. So uh, just do that? istream/ostream are standardized interfaces for reading and writing to "stuff", so that the library code doesn't have to care what "stuff" is.

For example if you have an asio::streambuf or whatever:

asio::streambuf buf {};

// Populate the buffer with data by some mechanism

std::istream is {&buf};
nbt::NBT data {is};

Or if you're using plain std::vectors, you could use the boost stream wrappers to get the correct interface

std::vector<char> vec;

// Populate the buffer with data by some mechanism

boost::interprocess::bufferstream bs {vec.data(), vec.size()};
nbt::NBT data {bs};

Everyone and their mom has their own buffer type, we use istream so I don't have to know about all those types and their interfaces. If there's something I'm missing here please explain.

[Wunkolo]

Everyone and their mom has their own buffer type, we use istream so I don't have to know about all those types and their interfaces.

std::span abstracts many continuous-memory buffer types without having to be aware of any specific buffer-type(Many types can cast automatically to it such as std::vector and std::array and c-style arrays and pretty much anything that implements .data() and .size(). It can turn any span of any type into std::span<std::byte> as well for "blob-of-data" operations). Being able to address into a continuous chunk of memory would be a faster paradigm as well for stuff like memory-mapped files than constantly seeking and reading from a stream abstraction.

[nickelpro]

We don't seek at all, beyond the normal advancement of gptr/pptr from doing reads/writes.

If you have a std::span you could wrap it in std::spanstream to use the correct interface. Any code I add here to support a raw std::span would be identical to the code generated by using std::spanstream. The work that needs to be done (tracking a cursor, ie gptr/pptr) is identical.

I'm open to maybe adding a templated set of serializers/deserializers so that you can avoid the virtual interface overhead, but I doubt that's significant. Might be worth benchmarking. In any case it would fall into the same pitfall of "you need to meet our concept of what a buffer is". In which case we would probably want to use something pseudo-standard like asio's DynamicBuffer_v2

[ToKiNoBug]

Happy to accept a PR, but I don't really understand. Any mechanism that can "read from a buffer" is going to be isomorphic to wrapping that underlying buffer type in an istream. So uh, just do that? istream/ostream are standardized interfaces for reading and writing to "stuff", so that the library code doesn't have to care what "stuff" is.

For example if you have an asio::streambuf or whatever:

asio::streambuf buf {};

// Populate the buffer with data by some mechanism

std::istream is {&buf};
nbt::NBT data {is};

Or if you're using plain std::vectors, you could use the boost stream wrappers to get the correct interface

std::vector<char> vec;

// Populate the buffer with data by some mechanism

boost::interprocess::bufferstream bs {vec.data(), vec.size()};
nbt::NBT data {bs};

Everyone and their mom has their own buffer type, we use istream so I don't have to know about all those types and their interfaces. If there's something I'm missing here please explain.

Thanks for you answer, I have understand your, and I have tried to use std::stringstream as out stream. But weird things happened when I tried to get the encoded data through ss.str().data() and its length through ss.str().size(). (here ss is an instantiation of std::stringstream)

In this way I managed to get the encoded data, but the first several bytes are incorrect. Maybe I'm not using std::stringstream correctly, but I don't know the correct way to get the encoded data from std::stringstream.

[nickelpro]

The tests all use std::stringstream as their encode target. If you have an example of nbt that's misbehaving or bugged please create a bug report and I'll take a look