OutpostUniverse / OP2Utility

C++ library for working with Outpost 2 related files and tasks.
MIT License
4 stars 0 forks source link

Extend Stream library to support serializing containers of non-trivial types #278

Open DanRStevens opened 5 years ago

DanRStevens commented 5 years ago

Instead of trying to copy the whole vector to the stream at once (only possible for non-trivial types), we could instead iterate over each element of the vector, and serialize the element individually. This is possible if there is an appropriate Write(T& object) method. Such is already the case for std::string, as the Stream library natively supports this. The code of course generalizes to any non-trivial type, given the existence of an appropriate serialization method.

Example vector write method for non-trivially-copyable objects:

  Write(const std::vector<T, A>& vector) {
    for (const auto element : vector) {
     writer.Write(element);  // Call custom serializer for element type
    }
  }

Example usage:

void f(Stream::Writer& writer) {
  std::vector<std::string> nonTrivial;
  // ...
  writer.Write(nonTrivial);
}

In the above example, the serialization code would loop over the vector's elements, and call the custom std::string serialization code for each element.

Combine this with the proposal in Issue #277, and it would also add support for other arbitrary non-trivial types, provided they supply an appropriate T::Write(Stream::Writer& writer) method.

DanRStevens commented 5 years ago

I was just thinking, the above method generalizes to any container type, even ones that use non-contiguous memory. The specific reference to std::vector can be removed, and replaced with a generic template type parameter. That could simplify the allocator A type parameter part as well.

I suppose technically you might want to guard the template with the notion of the container being iterable (and the element type being writable by the stream library). There's no built in type_trait check for iterable, but a solution can be found here: https://stackoverflow.com/questions/13830158/check-if-a-variable-is-iterable

DanRStevens commented 5 years ago

I came across void_t, which uses an example that checks for iterable: https://en.cppreference.com/w/cpp/types/void_t

Another point worth mentioning, which is touched on in that, is using cbegin and cend, rather than begin and end. The "c" prefix is for const, and produces a const iterator, which can not modify the object being iterated over. Reference: https://en.cppreference.com/w/cpp/iterator/begin


I came across std::data: https://en.cppreference.com/w/cpp/iterator/data

It allows for consistent treatment of containers with a .data() method, as well as arrays, and initializer lists. In terms of built in containers, and container like objects with .data(), that includes std::array, std::vector, and std::string. In particular, the method to write strings and vectors can be collapsed, as the method body is effectively the same.

More to the point here, I realized the only contiguous memory containers are likely to provide a .data() method. It wouldn't really make sense for other container types to provide such a member. That may lead to a useful check for the write method that assumes a container with contiguous memory.