Template the containers passed to archives

[pfeatherstone]

Feature request: Could the container passed to the archives be templated to give the developer the choice of using vectors instead of streams, or any other container.

[pfeatherstone]

Or have writer/reader base classes which are passed to the archivers and have derived classes for streams and vectors.

[pfeatherstone]

I've quickly modified PortableBinaryOutputArchive and PortableBinaryInputArchive to accept vector& and const vector& respectively and it seems to be a bit faster compared to using streams!

[uentity]

Can you please share the result of modification?

[pfeatherstone]

cereal_portable_binary_buf.txt

[pfeatherstone]

cereal_portable_binary_size.txt Also this so i can quickly get the size of the buffer without serialising

[pfeatherstone]

The changes are really minor

[pfeatherstone]

Actually don't worry about it. yas is faster

[memen45]

This would be a very useful feature. I am working on an embedded system and would like to avoid iostream if possible. I am wondering why the i/ostream has to be psased in the constructor:

// write to bin
cereal::BinaryOutputArchive oarchive(std::cout);
oarchive(myData);

//read from bin
cereal::BinaryInputArchive iarchive(std::cin);
iarchive(myData);

While I would expect something more C++ like

// write to bin
cereal::BinaryOutputArchive oarchive;
std::cout << oarchive(myData);

// read from bin
cereal::BinaryInputArchive iarchive;
std::cin >> iarchive(myData);

However, in the latter case, the SomeOutputArchive class does not need to have any data members, so static access would suffice. Operator() does not really make sense in that case, so it could be renamed as desired. Additional static functions can now be added as well:

// output to stream
std::cout << cereal::BinaryOutputArchive::getStream(myData);

// output to any compatible container using template
std::vector<uint8_t> binData = cereal::BinaryOutputArchive::archive<std::vector<uint8_t>>(myData);

// output using template deduction and pass by reference (more attractive than above)
cereal::BinaryOutputArchive::archive(myData, binData);

// input from stream
std::cin >> cereal::BinaryInputArchive::getStream(myData);

// input from any compatible container using template
std::vector<uint8_t> binData;
MyClass myData = cereal::BinaryInputArchive::archive<MyClass>(binData);

// input using template deduction and pass by reference (more attractive than above)
cereal::BinaryInputArchive::archive(binData, myData);

Backward compatibility (instantiation with the i/ostream) could be easily maintained, since the new functions will be static and can thus be used by an instantiated object as well and operator() may remain.

Any thoughts on this?

[memen45]

Okay, so I have an idea on how to implement this for all archives. I have forked this git to work on this and just pushed a first commit: https://github.com/memen45/cereal/commit/60d1c9cde07799f29434ded710ea91f632c7ed61.

The approach is relatively simple. PortableBinaryInputArchive and PortableBinaryOutputArchive classes were both templatized for the desired Container. Then all the sputn and sgetn function calls (specific to std::istream) were replaced by a simple std::copy_n. This function takes an InputIterator, a count and an OutputIterator. Then a bunch of meta templating makes sure the correct iterator type is selected:

std::istream_iterator<uint8_t>; // for InputArchive with istream
Container::const_iterator; // for InputArchive with any STL container
std::ostream_iterator<uint8_t>; // for OutputArchive with ostream
std::back_insert_iterator<Container>; // for OutputArchive with any STL container

This is basically all there is to it!

Is it possible for someone with knowledge on the code base to take a look at this commit memen45@60d1c9c to determine if this is a feasible approach for all Archive classes?