What happens at deserialization-time if the FsPickler version is different?

[brodyberg]

Let's say I am using FsPickler version 100. I pickle to Json text and the version 100 is written into the pickled output.

I deserialize with FsPickler version 100. Everything is fine.

But what happens when I deserialize with FsPickler 200? Will that happen successfully even though serialization was conducted by a different version of FsPickler?

Thanks,

Brody

[eiriktsarpalis]

Hi Brody,

The indicated version number is that of the FsPickler Json format, not the library version. The current (and only) format version is 0.9.6, since Json support was added in FsPickler 0.9.6. Future format revisions may or may not break deserialization of older data.

You can always omit header information from your pickled objects like this:

let jsp = FsPickler.CreateJson(omitHeader = true)

In general, FsPickler was not designed with version tolerance in mind. I would not recommend it for data storage unless you always plan to use the same version.

[Bananas-Are-Yellow]

I am planning to use FsPickler for data storage in binary format, because I want fast saving and loading, and compact data. There are two issues here:

1. The pickler format could change.
2. Your own data schema could change.

For (1), there needs to be a way to specify the FsPickler format version. Is this possible today, or does this mean that a specific version of FsPickler has to be used? I think I know the answer.

For (2), and this is not a question, just a comment, in my experience doing schema conversion as you deserialize is nasty. You need an object version number, so you know what format you are reading, and over time your code ends up with knowledge of all versions that ever existed, and it becomes an unmaintainable mess. And to do this with FsPickler would require every object to use a custom pickler for conversion, which defeats the idea of using automatically generated picklers, which is one of the attractions of FsPickler. Also, some schema changes are not simply to the fields inside an object, but require reorganization of objects.

My conclusion is that the best approach is to read in the old data using the old types and old deserialization code, and then convert it in memory to the new types. The old and new types can be kept in different namespaces, which indicate the overall schema number. Conversion code can be a separate program, so it doesn't need to pollute your serialization code.

David.

[dsyme]

Closing this old question