'Name = Value' archive

[DrAWolf]

Forgive me for posting another enhancement 'issue'...

I love to use a simple 'name = value' pair file for logging as well as for simple configuration files.

(My true motivation for switching from boost to cereal was this: I found it impossible (due to code complexity) to add such a custom archive to boost. The unavailability of a JSON archive for years tells the same story.)

The definition of nvp files is this:

• for simple types (arithmetic,string) a line is added:

    name = value

• for containers as array/vector/list square brackets are used

   container = [ a, b, c, d ]

• for complex objects (i.e. user defined classes) an object in braces is added:

   object =
   {
      name1 = value1 

      name2 = value2
      ...
   }

objects can be nested.

Having such an archive would also imply automatic "pretty printing" of std containers which is a common need.

Having such a simple bulit-in archive (to be used e.g. for config files and logging purposes) would be a distinguishing feature of your lib. It is - once more - telling, that boost included a simple log archive as an example in the docs, but that nobody was there to finish it.

I might post some code if I get this to work - at least for output

[AzothAmmo]

What would your output look like for various types of nested objects? What about a simple C style array of objects which themselves have a mix of fields including other objects?

Writing this archive is easy if you can completely define how you want it to look and come up with a good way to parse it. The only real difficulty in NVP based archives is adding the ability to support out of order loading.

[DrAWolf]

the following (incomplete) code should clarify the definition: nesting is almost the same as in JSON

I have not the ressources to write a parser atm, but it should not be too difficult.

/*! \file nvp.hpp
    \brief NVP input and output archives */
/*
  Copyright (c) 2013, Randolph Voorhies, Shane Grant
  All rights reserved.

  Redistribution and use in source and binary forms, with or without
  modification, are permitted provided that the following conditions are met:
  * Redistributions of source code must retain the above copyright
  notice, this list of conditions and the following disclaimer.
  * Redistributions in binary form must reproduce the above copyright
  notice, this list of conditions and the following disclaimer in the
  documentation and/or other materials provided with the distribution.
  * Neither the name of cereal nor the
  names of its contributors may be used to endorse or promote products
  derived from this software without specific prior written permission.

  THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND
  ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED
  WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE
  DISCLAIMED. IN NO EVENT SHALL RANDOLPH VOORHIES OR SHANE GRANT BE LIABLE FOR ANY
  DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES
  (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES;
  LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND
  ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
  (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS
  SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
  */
#ifndef CEREAL_ARCHIVES_NVP_HPP_
#define CEREAL_ARCHIVES_NVP_HPP_

#include <cereal/cereal.hpp>
#include <cereal/details/util.hpp>

#include <sstream>
#include <stack>
#include <vector>
#include <string>

namespace cereal
{
    // ######################################################################
    //! An output archive designed to save data to name = value pairs (NVP)
    /*!

        \ingroup Archives */

    class NVPOutputArchive : public OutputArchive<NVPOutputArchive>
    {
        enum class NodeType { Default, StartObject, InObject, StartArray, InArray };

    public:
        //! Construct, outputting to the provided stream
        /*! @param stream The stream to output to.  Can be a stringstream, a file stream, or
                          even cout!
                          @param precision The precision for floating point output */
        NVPOutputArchive( std::ostream & stream, int precision = 20 ) :
            OutputArchive<NVPOutputArchive>( this ),
            itsWriteStream( stream ),
            itsNextName( nullptr ),
            itsDepth( 0 )
        {
            itsNameCounter.push( 0 );
            itsNodeStack.push( NodeType::Default );  // StartObject results in an unwanted { + newline for 
            }

        //! Destructor, flushes the stream
        ~NVPOutputArchive()
        {
            itsWriteStream.flush();
        }

        //! saving values: just pass to stream       
        template<class T>
        void saveValue( T t )
        {
            itsWriteStream << t;
        }

        //! saving chars as ints - otherwise they appear as ascii characters in text file      
        template<>
        void saveValue( unsigned char c )
        {
            itsWriteStream << (uint)c;
        }

        template<>
        void saveValue( char c )
        {
            itsWriteStream << (int)c;
        }

        //! saving strings: use quotes       
        template<>
        void saveValue( std::string s )
        {
            itsWriteStream << "\"" << s << "\"";
        }

        void doIndent()
        {
            for ( int i = 0; i < itsDepth; ++i )
            {
                itsWriteStream << "    ";
            }
        }

        void saveName( const std::string& name )
        {
            doIndent();
            itsWriteStream << name;
            itsWriteStream << " = ";
        }

        void StartArray()
        {
            itsWriteStream << "[";
        }

        void EndArray()
        {
            itsWriteStream << "]";

            itsNodeStack.pop();
            itsNameCounter.pop();

        }

        void StartObject()
        {
            if ( true ) // style 1
            {
                itsWriteStream << "\n";
                doIndent();
                itsWriteStream << "{\n";
                ++itsDepth;
            }
            else  // style 2
            {
                itsWriteStream << "{\n";
                ++itsDepth;
            }
        }

        void EndObject()
        {
            --itsDepth;
            if ( itsNodeStack.top() == NodeType::InObject )
            {
                itsWriteStream << "\n";
            }
            doIndent();
            itsWriteStream << "}";

            itsNodeStack.pop();
            itsNameCounter.pop();
        }

#ifdef _MSC_VER
        // Visual Studio has problems disambiguating the above for unsigned long, so we provide an explicit
        // overload for long and serialize it as its size necessitates
        //
        // When loading we don't need to do this specialization since we catch the types with
        // templates according to their size

        //! 32 bit long saving
        template <class T> inline
            typename std::enable_if<sizeof( T ) == sizeof( std::uint32_t ), void>::type
            saveLong( T lu ) { saveValue( static_cast<std::uint32_t>( lu ) ); }

        //! non 32 bit long saving
        template <class T> inline
            typename std::enable_if<sizeof( T ) != sizeof( std::uint32_t ), void>::type
            saveLong( T lu ) { saveValue( static_cast<std::uint64_t>( lu ) ); }

        //! MSVC only long overload
        void saveValue( unsigned long lu ) { saveLong( lu ); };
#endif

        //! Save exotic arithmetic types as binary
        //template<class T>
        //typename std::enable_if<std::is_arithmetic<T>::value &&
        //    ( sizeof( T ) >= sizeof( long double ) || sizeof( T ) >= sizeof( long long ) ), void>::type
        //    saveValue( T const & t )
        //{
        //    auto base64string = base64::encode( reinterpret_cast<const unsigned char *>( &t ), sizeof( T ) );
        //    saveValue( base64string );
        //    }

        //! Write the name of the upcoming node and prepare object/array state
        /*! Since writeName is called for every value that is output, regardless of
            whether it has a name or not, it is the place where we will do a deferred
            check of our node state and decide whether we are in an array or an object. */
        void writeName()
        {
            NodeType const & nodeType = itsNodeStack.top();

            // Start up either an object or an array, depending on state
            if ( nodeType == NodeType::InArray )
            {
                itsWriteStream << ", ";
            }
            else if ( nodeType == NodeType::InObject )
            {
                itsWriteStream << "\n";         // new names on new line
            }

            if ( nodeType == NodeType::StartArray )
            {
                StartArray();
                itsNodeStack.top() = NodeType::InArray;
            }
            else if ( nodeType == NodeType::StartObject )
            {
                StartObject();
                itsNodeStack.top() = NodeType::InObject;
            }

            // Array types do not output names
            if ( nodeType == NodeType::InArray )
            {
                return;
            }

            if ( itsNextName == nullptr )
            {
                std::string name = "value" + std::to_string( itsNameCounter.top()++ ) + "\0";
                doIndent();
                itsWriteStream << name;
                itsWriteStream << " = ";
            }
            else
            {
                doIndent();
                itsWriteStream << itsNextName;
                itsWriteStream << " = ";
                itsNextName = nullptr;
            }

        }

        //! Starts a new node in the NVP output
        void startNode()
        {
            writeName();
            itsNodeStack.push( NodeType::StartObject );
            itsNameCounter.push( 0 );
        }

        //! Designates the most recently added node as finished
        void finishNode()
        {
            const NodeType node_type = itsNodeStack.top();
            // if we ended up serializing an empty object or array, writeName
            // will never have been called - so start and then immediately end
            // the object/array.
            //
            // We'll also end any object/arrays we happen to be in

            switch ( node_type )
            {
                case NodeType::StartArray:
                    StartArray();
                case NodeType::InArray:
                    EndArray();
                    break;
                case NodeType::StartObject:
                    StartObject();
                case NodeType::InObject:
                    EndObject();
                    break;
            }

        }

        //! Designates that the current node should be output as an array, not an object
        void makeArray()
        {
            itsNodeStack.top() = NodeType::StartArray;
        }

        //! Sets the name for the next node created with startNode
        void setNextName( const char * name )
        {
            itsNextName = name;
        }

        //! Saves some binary data, encoded as a base64 string, with an optional name
        /*! This will create a new node, optionally named, and insert a value that consists of
            the data encoded as a base64 string */
        //void saveBinaryValue( const void * data, size_t size, const char * name = nullptr )
        //{
        //    setNextName( name );
        //    writeName();

        //    auto base64string = base64::encode( reinterpret_cast<const unsigned char *>( data ), size );
        //    saveValue( base64string );
        //};

    private:
        std::ostream& itsWriteStream;          //!<  write stream
        char const * itsNextName;            //!< The next name
        std::stack<uint32_t> itsNameCounter; //!< Counter for creating unique names for unnamed nodes
        std::stack<NodeType> itsNodeStack;
        int itsDepth;
    }; // NVPOutputArchive

    // ######################################################################
    //! An input archive designed to load data from JSON
    /*! This archive uses RapidJSON to read in a JSON archive.

        Input JSON should have been produced by the NVPOutputArchive.  Data can
        only be added to dynamically sized containers (marked by JSON arrays) -
        the input archive will determine their size by looking at the number of child nodes.

        The order of the items in the JSON archive must match what is expected in the
        serialization functions.

        \ingroup Archives */
    class NVPInputArchive : public InputArchive<NVPInputArchive>
    {
        //    typedef rapidjson::GenericReadStream ReadStream;
        //    typedef rapidjson::GenericValue<rapidjson::UTF8<>> JSONValue;
        //    typedef JSONValue::ConstMemberIterator MemberIterator;
        //    typedef JSONValue::ConstValueIterator ValueIterator;
        //    typedef rapidjson::Document::GenericValue GenericValue;

        //    //! An internal iterator that handles both array and object types
        //    class Iterator
        //    {
        //    public:
        //        Iterator() : itsType( Null ) {}

        //        Iterator( MemberIterator it ) :
        //            itsMemberIt( it ), itsType( Member )
        //        {
        //        }

        //        Iterator( ValueIterator it ) :
        //            itsValueIt( it ), itsType( Value )
        //        {
        //        }

        //        Iterator & operator++( )
        //        {
        //            switch ( itsType )
        //            {
        //                case Value: ++itsValueIt; break;
        //                case Member: ++itsMemberIt; break;
        //                default: throw cereal::Exception( "Invalid Iterator Type!" );
        //            }
        //            return *this;
        //        }

        //        GenericValue const & value()
        //        {
        //            switch ( itsType )
        //            {
        //                case Value: return *itsValueIt;
        //                case Member: return itsMemberIt->value;
        //                default: throw cereal::Exception( "Invalid Iterator Type!" );
        //            }
        //        }

        //    private:
        //        MemberIterator itsMemberIt;
        //        ValueIterator itsValueIt;
        //        enum Type { Value, Member, Null } itsType;
        //    };

    public:
        //! Construct, outputting to the provided stream
        /*! @param stream The stream to output to.  Can be a stringstream, a file stream, or
                          even cout! */
        NVPInputArchive( std::istream & stream ) :
            InputArchive<NVPInputArchive>( this ),
            itsReadStream( stream )
        {
            //itsDocument.ParseStream<0>( itsReadStream );
            //itsValueStack.push_back( itsDocument.MemberBegin() );
            }

        //! Starts a new node, going into its proper iterator
        void startNode()
        {
            //if ( itsValueStack.back().value().IsArray() )
            //    itsValueStack.push_back( itsValueStack.back().value().Begin() );
            //else
            //    itsValueStack.push_back( itsValueStack.back().value().MemberBegin() );
        }

        //! Finishes the most recently started node
        void finishNode()
        {
            //itsValueStack.pop_back();
            //++itsValueStack.back();
        }

        //    template<class T>
        //    typename std::enable_if < std::is_signed<T>::value && sizeof( T ) < sizeof( int64_t ), void>::type
        //        loadValue( T & val )
        //    {
        //        val = itsValueStack.back().value().GetInt();
        //        ++itsValueStack.back();
        //        }

        //    template<class T>
        //    typename std::enable_if < ( std::is_unsigned<T>::value && sizeof( T ) < sizeof( uint64_t ) ) &&
        //        !std::is_same<bool, T>::value, void>::type
        //        loadValue( T & val )
        //    {
        //        val = itsValueStack.back().value().GetUint();
        //        ++itsValueStack.back();
        //        }

        //    void loadValue( bool & val ) { val = itsValueStack.back().value().GetBool();   ++itsValueStack.back(); }
        //    void loadValue( int64_t & val ) { val = itsValueStack.back().value().GetInt64();  ++itsValueStack.back(); }
        //    void loadValue( uint64_t & val ) { val = itsValueStack.back().value().GetUint64(); ++itsValueStack.back(); }
        //    void loadValue( float & val ) { val = static_cast<float>( itsValueStack.back().value().GetDouble() ); ++itsValueStack.back(); }
        //    void loadValue( double & val ) { val = itsValueStack.back().value().GetDouble(); ++itsValueStack.back(); }
        //    void loadValue( std::string & val ) { val = itsValueStack.back().value().GetString(); ++itsValueStack.back(); }

        //    template<class T>
        //    typename std::enable_if<std::is_arithmetic<T>::value &&
        //        ( sizeof( T ) >= sizeof( long double ) || sizeof( T ) >= sizeof( long long ) ), void>::type
        //        loadValue( T & val )
        //    {
        //        std::string encoded;
        //        loadValue( encoded );
        //        auto decoded = base64::decode( encoded );

        //        if ( sizeof( T ) != decoded.size() )
        //            throw Exception( "Decoded binary data size does not match specified size" );

        //        std::memcpy( &val, decoded.data(), decoded.size() );
        //        }

        //    //! Loads some binary data, encoded as a base64 string
        //    void loadBinaryValue( void * data, size_t size )
        //    {
        //        std::string encoded;
        //        loadValue( encoded );
        //        auto decoded = base64::decode( encoded );

        //        if ( size != decoded.size() )
        //            throw Exception( "Decoded binary data size does not match specified size" );

        //        std::memcpy( data, decoded.data(), decoded.size() );
        //    };

        //    //! Loads the size for a SizeTag
        //    void loadSize( size_type & size )
        //    {
        //        size = ( itsValueStack.rbegin() + 1 )->value().Size();
        //    }

    private:
        std::istream& itsReadStream;               //!< Rapidjson write stream
        //    std::vector<Iterator> itsValueStack;    //!< Stack of values
        //    rapidjson::Document itsDocument;        //!< Rapidjson document
    };

    // ######################################################################
    // JSONArchive prologue and epilogue functions
    // ######################################################################

    // ######################################################################
    //! Prologue for NVPs for JSON archives
    /*! NVPs do not start or finish nodes - they just set up the names */
    template <class T>
    void prologue( NVPOutputArchive &, NameValuePair<T> const & )
    {
    }

    //! Prologue for NVPs for JSON archives
    template <class T>
    void prologue( NVPInputArchive &, NameValuePair<T> const & )
    {
    }

    // ######################################################################
    //! Epilogue for NVPs for JSON archives
    /*! NVPs do not start or finish nodes - they just set up the names */
    template <class T>
    void epilogue( NVPOutputArchive &, NameValuePair<T> const & )
    {
    }

    //! Epilogue for NVPs for JSON archives
    /*! NVPs do not start or finish nodes - they just set up the names */
    template <class T>
    void epilogue( NVPInputArchive &, NameValuePair<T> const & )
    {
    }

    // ######################################################################
    //! Prologue for SizeTags for JSON archives
    /*! SizeTags are strictly ignored for JSON */
    template <class T>
    void prologue( NVPOutputArchive & ar, SizeTag<T> const & )
    {
        ar.makeArray();
    }

    //! Prologue for SizeTags for JSON archives
    template <class T>
    void prologue( NVPInputArchive &, SizeTag<T> const & )
    {
    }

    // ######################################################################
    //! Epilogue for SizeTags for JSON archives
    /*! SizeTags are strictly ignored for JSON */
    template <class T>
    void epilogue( NVPOutputArchive &, SizeTag<T> const & )
    {
    }

    //! Epilogue for SizeTags for JSON archives
    template <class T>
    void epilogue( NVPInputArchive &, SizeTag<T> const & )
    {
    }

    // ######################################################################
    //! Prologue for all other types for JSON archives
    /*! Starts a new node, named either automatically or by some NVP,
        that may be given data by the type about to be archived */
    template <class T>
    typename std::enable_if<!std::is_arithmetic<T>::value && !std::is_enum<T>::value && !cereal::is_primitive<T>::value&& !std::is_pointer<T>::value, void>::type
        prologue( NVPOutputArchive & ar, T const & )
    {
        ar.startNode();
        }

    //! Prologue for all other types for JSON archives
    template <class T>
    typename std::enable_if<!std::is_arithmetic<T>::value && !std::is_enum<T>::value && !cereal::is_primitive<T>::value, void>::type
        prologue( NVPInputArchive & ar, T const & )
    {
        ar.startNode();
        }

    // ######################################################################
    //! Epilogue for all other types other for JSON archives
    /*! Finishes the node created in the prologue */
    template <class T>
    typename std::enable_if<!std::is_arithmetic<T>::value && !std::is_enum<T>::value && !cereal::is_primitive<T>::value, void>::type
        epilogue( NVPOutputArchive & ar, T const & )
    {
        ar.finishNode();
        }

    //! Epilogue for all other types other for JSON archives
    template <class T>
    typename std::enable_if<!std::is_arithmetic<T>::value && !std::is_enum<T>::value && !cereal::is_primitive<T>::value, void>::type
        epilogue( NVPInputArchive & ar, T const & )
    {
        ar.finishNode();
        }

    // ######################################################################
    //! Prologue for arithmetic types for JSON archives
    template <class T>
    typename std::enable_if<std::is_arithmetic<T>::value /*|| std::is_enum<T>::value || cereal::is_primitive<T>::value*/, void>::type
        prologue( NVPOutputArchive & ar, T const & )
    {
        ar.writeName();
        }

    //! Prologue for arithmetic types for JSON archives
    template <class T>
    typename std::enable_if< std::is_arithmetic<T>::value /*|| std::is_enum<T>::value || cereal::is_primitive<T>::value*/, void >::type
        prologue( NVPInputArchive &, T const & )
    {
        }

    // ######################################################################
    //! Epilogue for arithmetic types for JSON archives
    template <class T>
    typename std::enable_if<std::is_arithmetic<T>::value, void>::type
        epilogue( NVPOutputArchive &, T const & )
    {
        }

    //! Epilogue for arithmetic types for JSON archives
    template <class T>
    typename std::enable_if<std::is_arithmetic<T>::value, void>::type
        epilogue( NVPInputArchive &, T const & )
    {
        }

    // ######################################################################
    //! Prologue for strings for JSON archives
    template<class CharT, class Traits, class Alloc> inline
        void prologue( NVPOutputArchive & ar, std::basic_string<CharT, Traits, Alloc> const & )
    {
        ar.writeName();
        }

    //! Prologue for strings for JSON archives
    template<class CharT, class Traits, class Alloc> inline
        void prologue( NVPInputArchive &, std::basic_string<CharT, Traits, Alloc> const & )
    {
        }

    // ######################################################################
    //! Epilogue for strings for JSON archives
    template<class CharT, class Traits, class Alloc> inline
        void epilogue( NVPOutputArchive &, std::basic_string<CharT, Traits, Alloc> const & )
    {
        }

    //! Epilogue for strings for JSON archives
    template<class CharT, class Traits, class Alloc> inline
        void epilogue( NVPInputArchive &, std::basic_string<CharT, Traits, Alloc> const & )
    {
        }

    // ######################################################################
    // Common JSONArchive serialization functions
    // ######################################################################

    //! Serializing NVP types to JSON
    template <class T> inline
        void save( NVPOutputArchive & ar, NameValuePair<T> const & t )
    {
        ar.setNextName( t.name );
        ar( t.value );
        }

    template <class T> inline
        void load( NVPInputArchive & ar, NameValuePair<T> & t )
    {
        ar( t.value );
        }

    //! Saving for arithmetic to JSON
    template<class T> inline
        typename std::enable_if<std::is_arithmetic<T>::value, void>::type
        save( NVPOutputArchive & ar, T const & t )
    {
        ar.saveValue( t );
        }

    //! Loading arithmetic from JSON
    template<class T> inline
        typename std::enable_if<std::is_arithmetic<T>::value, void>::type
        load( NVPInputArchive & ar, T & t )
    {
        //ar.loadValue( t );
        }

    //! saving string to JSON
    template<class CharT, class Traits, class Alloc> inline
        void save( NVPOutputArchive & ar, std::basic_string<CharT, Traits, Alloc> const & str )
    {
        ar.saveValue( str );
        }

    //! loading string from JSON
    template<class CharT, class Traits, class Alloc> inline
        void load( NVPInputArchive & ar, std::basic_string<CharT, Traits, Alloc> & str )
    {
        //ar.loadValue( str );
        }

    // ######################################################################
    //! Saving SizeTags to JSON
    template <class T> inline
        void save( NVPOutputArchive &, SizeTag<T> const & )
    {
        }

    //! Loading SizeTags from JSON
    template <class T> inline
        void load( NVPInputArchive & ar, SizeTag<T> & st )
    {
        //ar.loadSize( st.size );
        }

} // namespace cereal

// register archives for polymorphic support
CEREAL_REGISTER_ARCHIVE( cereal::NVPInputArchive );
CEREAL_REGISTER_ARCHIVE( cereal::NVPOutputArchive );

#endif // CEREAL_ARCHIVES_NVP_HPP_