TensorCollection metadata serialization using Cereal

[dmccloskey]

Description

As a user, I would like to store the TensorCollection metadata in a binary form that has minimal disk space and is fast to read and write to.

Objectives

• [ ] Add Cereal archive decorators to all classes that contain metadata
• [ ] Add a TensorCollectionFile class that reads/writes the cereal archives
• [ ] Make unit tests covering the read and write operations

Validation

• [ ] Unit test coverage
• [ ] Working win/linux builds

Code Snippets

storeTensorCollectionBinary with e.g. (const std::string & filename, const InterpreterT& model_interpreter) arguments (To be updated)

        std::ofstream ofs(filename, std::ios::binary);  
        //if (ofs.is_open() == false) {// Lines check to make sure the file is not already created
        cereal::BinaryOutputArchive oarchive(ofs); 
        oarchive(model_interpreter); 
        ofs.close();
        //}// Lines check to make sure the file is not already created
        return true;

loadTensorCollectionBinary with e.g. (const std::string & filename, InterpreterT& model_interpreter) arguments (To be updated)

        std::ifstream ifs(filename, std::ios::binary); 
        if (ifs.is_open()) {
            cereal::BinaryInputArchive iarchive(ifs);
            iarchive(model_interpreter);
            ifs.close();
        }
        return true;

e.g. class headers

#include <cereal/access.hpp>  // serialiation of private members
#include <cereal/types/memory.hpp>
#include <cereal/types/map.hpp>
#include <cereal/types/utility.hpp> // std::pair
#include <cereal/types/vector.hpp>

e.g. class decorators

private:
        friend class cereal::access;
        template<class Archive>
        void serialize(Archive& archive)
        {
            archive(id_, name_, module_id_, module_name_, layer_name_, output_max_, output_min_,
                tensor_index_, type_, status_, 
                activation_, activation_grad_, 
                integration_, integration_error_, integration_weight_grad_
            );
        }