Serialization

[tlepoint]

Serialization of the poly and poly_p using the (optional) cereal library (https://github.com/USCiLab/cereal) (if the library is not present, NFLlib still compiles and successfully tests).

I considered using boost rather than cereal, but either it requires to add boost as a dependency to add the serialize function (which I don't want), or I could not get the non intrusive version to work, possibly because the class does not expose "_data" to reconstruct the class state.

Note that:

• the master branch of cereal is not active but the development branch is;
• libcereal-dev is not in precise (Ubuntu), but is in xenial (Ubuntu 16.04LTS http://packages.ubuntu.com/xenial/libcereal-dev), so I had to use a git clone command

What do you suggest?

1. We could not use cereal, but then I need help with boost.
2. Is there a way to use xenial in travis and to add libcereal-dev as a package in the .travis.yml file?
3. The current solution works well enough.

[carlosaguilarmelchor]

Nice work Tancrède. Have you tested the compactness/speed of cereal ? (to compare with what we would obtain by doing the serialization manually, as in practice you only save _data)

Correct me if I am wrong but imho the tests should :

• serialize p0 to Arch
• copy p0 to p0'
• free p0 (to avoid false negatives on the tests due to pointers)
• déserialize from Arch p1
• test that p1 = p0'

Carlos

[serge-sans-paille]

hi tancrede. I don't have any personnal experience with cereal,. i did use boost.serialize in the past and it was ok. I also received good feedback on cereal lastly, so no strong opinion there.

Why do you need serialization by the way? i'll review your post later ;-)

[serge-sans-paille]

btw +1 for the tests @carlosaguilarmelchor proposes

[tlepoint]

I modified the test file as suggested.

I didn't try to do a manual serialization, I liked the C++ way of doing things with cereal, and I am not sure how to do it properly by hand. Do you think we should do it by hand?

I think serialization is a nice addition to the library (the overhead in term of code being minimal) and will be useful in applications (e.g. in FV-NFLlib, I would like to serialize ciphertexts, public/secret keys, etc.).

[carlosaguilarmelchor]

I suppose it is better to do it with cereal but as it may be used for sending ciphertexts over the network and have an influence on performance measures it would be nice to know that we are not paying too much for it.

[tlepoint]

On my computer (on battery), I have the following results

• 8_60_uint32_t

Cereal output serialization: 0.286 us
Cereal input serialization: 0.304 us
Manual output serialization: 0.196 us
Manual input serialization: 0.202 us

• 512_30_uint32_t

Cereal output serialization: 10.04 us
Cereal input serialization: 6.156 us
Manual output serialization: 4.518 us
Manual input serialization: 4.514 us

• 1024_60_uint32_t

Cereal output serialization: 32.526 us
Cereal input serialization: 31.886 us
Manual output serialization: 18.036 us
Manual input serialization: 17.256 us

• 8192_124_uint64_t

Cereal output serialization: 572.434 us
Cereal input serialization: 373.856 us
Manual output serialization: 354.85 us
Manual input serialization: 319.474 us

What do you think? We could use the manual functions, the code is eventually simple.

[aguinet]

The code looks fine for me, I agree with @carlosaguilarmelchor for the behavior of the tests ;) I've got a question about your serailisation : you just need to output a bucnh of bytes, and "upper"l ayers know which type of polynom you have ? If you indeed just have a fixed size array to write back and read forth, use your manual version :)

[tlepoint]

Just updated to use manual functions instead of cereal (since we need the parameters at compilation, it makes sense to only save _data). Is that OK with everyone?

[carlosaguilarmelchor]

@tlepoint saving only _data is perfectly ok with me.

I still have two questions tough.

1) Did you run the tests for libcereal with a binary archive ? (it is the closest to the manual serialization and they say it is the fastest) If this is not the case can you rerun the tests with a binary and portable_binary archive ?

2) Is the manual serialization compatible with libcereal and/or boost:serialize ? I mean, if there is an object O with a poly_t as an attribute and O is to complex to serialize manually will libcereal/boost be able to use the serialization functions we defined manually for the poly_t ?

Thx for the nice work btw :)

[tlepoint]

1) Yes I used cereal::BinaryOutputArchive and cereal::BinaryInputArchive 2) No it will not be compatible with libcereal in the sense you're suggesting. The object O will have to implement the serialize() function, but if it contains a polynomial then I think you'll have to reinterpret_cast the begin() as an array to save it with cereal because poly_t does not implement libcereal serialize function. That's why I wanted to use boost or libcereal in the first place.

[carlosaguilarmelchor]

In fact my question was : "isn't there a way to do the manual serialization in a way that it will be reused by libcereal/boost if someone uses them for higher level objects ?"

For portability I don't see any other possibly issue than endianness. Given that little-endian is present on any platform in which we may imagine NFLlib used I think it is enough to do as @serge-sans-paille proposed: saying on the doc of the functions that serialization should not be used in the rare cases a big-endian machine will use NFLlib and communicate serialized data to a little-endian machine.

[tlepoint]

In fact my question was : "isn't there a way to do the manual serialization in a way that it will be reused by libcereal/boost if someone uses them for higher level objects ?"

I don't think so.

[carlosaguilarmelchor]

What about : http://uscilab.github.io/cereal/archive_specialization.html ? I have the feeling it is possible to overload what libcereal does and have all the benefits but I am unsure of how to do it.

[carlosaguilarmelchor]

+1 for merge

[tlepoint]

@serge-sans-paille @aguinet Is it OK for you?