Preferred method to deal with non-owning pointers

[alastairUK]

cereal doesn't support "Raw/Dumb Pointers and References" so I am wondering what the preferred method is for dealing with non-owning pointers. I could use a shared_ptr and weak_ptr but this seems like a misuse of shared_ptr in my use case.

I do like https://github.com/martinmoene/observer-ptr (based on http://open-std.org/jtc1/sc22/wg21/docs/papers/2014/n3840.pdf) to make it clear what the pointer is being used for - how easy would it be for me to support this "smart" pointer type in cereal?

[AzothAmmo]

With smart owning pointers, we can immediately perform the load and properly set up the pointer, or refer to a prior load of the same pointer in the case of shared pointers. The problem with normal pointers is that we can't efficiently keep track of whether the data has already been saved or loaded because this requires recording and checking the addresses of everything we serialize.

Even in the case of a non-owning pointer, there are still problems. Since cereal serializes data in a one-pass depth-first fashion, it has no knowledge of the position of future data, and cannot properly handle the pointer in a general sense. cereal would essentially need to serialize something like the offset of the pointer (within the serialized data) and use this information when loading, in conjunction with keeping track of addresses, to restore the pointer.

If cereal had a two-pass serialization step, the location of all data would be known before serialization, which would enable non-owning pointers to be supported. I've written some thoughts on this at #185.

That being said you can indeed write support for this but you need to essentially solve the problem of non-smart pointers.

[cstamatopoulos]

In a similar manner to the cereal serialization of shared_ptr<>, it is possible to support the upcoming observer_ptr<> but only for unique_ptr<>. It could be done for shared_ptr<> as well but I assume one would use weak_ptr in that case and would require an extra parameter to state this is an observer_ptr to a shared_ptr which would be user error-prone.

It is also possible to assert in the destructor of the serialized / deserialized observer_ptr objects all have a unique_ptr "parent". Not that you can do much since the check only happens in the end but at least helps in debugging when you accidentally add an observer_ptr to say a non unique_ptr object.

If there is any interest I can provide my code but it does break the current serialization format for unique_ptr. No major change but still a change. I have implemented it using the observer_ptr from https://github.com/martinmoene/observer-ptr